TwelveLabsのエンベディングとQdrantの検索機能を使用した、ビデオセマンティック推薦システムの構築

u30bfu30b0u3001u30bfu30b3u30c8u30cfu3001u30adu30fcu30efu30fcu30c9u306bu305fu3088u308bu3053u3068u306au304fu3001u30a3u30c7u30aau306eu4e2du8eabu3092u672cu5f53u306bu7406u89e3u3067u304du308bu30b3u30f3u30c6u30f3u30c4u30ecu30b3u30e1u30f3u30c9u30b7u30b9u30c6u30e0u304cu3042u3063u305fu3089u3069u3046u3067u3057u30e7u3046u304buff1f ud83dudd0d

u3053u306eu30c1u30e5u30fcu30c8u30eau30a2u30ebu3067u306fu3001u30d3u30c7u30aau30b3u30f3u30c6u30f3u30c4u3092u610fu5473u30ecu30d9u30ebu3067u7406u89e3u3059u308au30ecu30b3u30e1u30f3u30c9u30b7u30b9u30c6u30e0u3092u69cbu7bc9u3057u307eu3059u3002TwelveLabs u306eu30d3u30c7u30aau57cbu3081u8fbcu307fu6a5fu80fdu3068 Qdrant u30d9u30afu30c8u30ebu985eu4f3cu5ea6u691cu7d22u3092u7d44u307fu5408u308fu305bu308bu3053u3068u3067u3001u5358u306au308bu30adu30fcu30efu30fcu30c9u306eu4e00u81f4u3060u3051u3067u306au304fu3001u5b9fu969bu306eu610fu5473u306bu57fau3065u3044u3066u95a2u90a3u3059u308bu30d3u30c7u30aau3092u898bu3064u3051u308bu30a8u30f3u30b8u30f3u3092u4f5cu6210u3057u307eu3059u3002

u3053u306eu30a2u30d7u30eau30b1u30fcu30b7u30e7u30f3u304cu3069u306eu3088u3046u306bu52d5u4f5cu3059u308bu306eu304bu3001u305du3057u3066TwelveLabs Python SDKu3068Qdrant Cloud Quickstartu3092u4f7fu3063u3066u540cu69d8u306eu30bdu30eau30e5u30fcu30b7u30e7u30f3u3092u3069u306eu3088u3046u306bu69cbu7bc9u3067u304du308bu304bu3092u63a2u3063u3066u307fu307eu3057u30e7u3046u3002

u30a2u30d7u30eau30b1u30fcu30b7u30e7u30f3u306eu30c7u30e2u306fu3053u3061u3089u3067u3054u89a7u3044u305fu3060u3051u307eu3059uff1a TwelveLabs u30b3u30f3u30c6u30f3u30c4u30ecu30b3u30e1u30f3u30c7u30fcu30b7u30e7u30f3

u524du63d0u6761u4ef6

• TwelveLabs Playgroundu3067u30b5u30a4u30f3u30a2u30c3u30d7u3057u3066u3001APIu30adu30fcu3092u767au884cu3057u307eu3059u3002

• u30bbu30c3u30c8u30a2u30c3u30d7u30acu30a4u30c9u306bu5f93u3063u3066u3001Qdrant Cloudu3092u30bbu30c3u30c8u30a2u30c3u30d7u3057u307eu3059u3002

• Githubu3067u3053u306eu30a2u30d7u30eau30b1u30fcu30b7u30e7u30f3u306eu30eau30ddu30b8u30c8u30eau3092u898bu3064u3051u307eu3059u3002

• Pythonu3001Flasku3001u304au3088u3073Next.jsu306bu7cbeu901au3057u3066u3044u308bu3053u3068u304cu671bu307eu3057u3044u3067u3059u3002

u30c7u30e2u30a2u30d7u30eau30b1u30fcu30b7u30e7u30f3

u3053u306eu30c7u30e2u30a2u30d7u30eau30b1u30fcu30b7u30e7u30f3u306fu3001u610fu5473u7684u306au30d3u30c7u30aau30b3u30f3u30c6u30f3u30c4u30ecu30b3u30e1u30f3u30c7u30fcu30b7u30e7u30f3u306eu529bu3092u793au3057u3066u3044u307eu3059u3002u30e6u30fcu30b6u30fcu304cu610fu56f3u306bu57fau3065u304fu30abu30c6u30b4u30eau3092u9078u629eu3059u308bu3068u3001u30b3u30b9u30e1u30c6u30a3u30c3u30afu985eu4f3cu5ea6u30deu30c3u30c1u30f3u30b0u3092u901au3058u3066u3001u30ecu30b3u30e1u30f3u30c9u30b7u30b9u30c6u30e0u304cu95a2u90a3u3059u308bu30b3u30f3u30c6u30f3u30c4u3092u63d0u6848u3057u307eu3059u3002u307eu305fu3001u30e6u30fcu30b6u30fcu304cu81eau5206u306eu6c17u5206u3092u6307u5b9au3057u3066u3001u81eau5206u306eu611fu60c5u306bu5408u3063u305fu30b3u30f3u30c6u30f3u30c4u3092u53d7u3051u53d6u308bu3053u3068u3082u3067u304du307eu3059u3002

u3053u306eu30b9u30c6u30e0u306fu3001Qdrant Cloudu306bu304au3051u308bu3055u307eu3056u307eu306au30abu30c6u30b4u30eau306eu30a2u30cbu30e1u52d5u753bu306eu5927u898fu6a21u306au30b3u30ecu30afu30b7u30e7u30f3u304bu3089u306eu57cbu3081u8fbcu307fu3092u4fddu5b58u3057u3066u3044u307eu3059u3002

u30a2u30d7u30eau30b1u30fcu30b7u30e7u30f3u306eu52d5u4f5cu30d7u30edu30bbu30b9

u30a2u30d7u30eau30b1u30fcu30b7u30e7u30f3u306fu4e3bu306b2u3064u306eu6 stageu3067u52d5u4f5cu3057u307eu3059uff1a

• u57cbu3081u8fbcu307fu306eu751fu6210u3068u4fddu5b58 u2013 u30b7u30b9u30c6u30e0u306fMarengo 2.7u3092u4f7fu7528u3057u3066u30d3u30c7u30aau57cbu3081u8fbcu307fu3092u4f5cu6210u3057u3001S3u30d0u30b1u30c3u30c8u3092u4ecbu3057u3066u516cu958bURLu3092u751fu6210u3057u307eu3059u3002u3053u308cu3089u306eu57cbu3081u8fbcu307fu3068u305du306eu30e1u30bfu30c7u30fcu30bfu306fu3001Qdrant Cloudu306eu30d9u30afu30c8u30ebu30c7u30fcu30bfu30d9u30fcu30b9u306eu30b3u30ecu30afu30b7u30e7u30f3u306bu4fddu5b58u3055u308cu307eu3059u3002

• u691cu7d22u3068u53d6u5f97 - u30e6u30fcu30b6u30fcu304cu597du307fu3084u691cu7d22u30afu30a8u30eau3092u5165u529bu3059u308bu3068u3001u30b3u30f3u30c6u30f3u30c4u30ecu30b3u30e1u30f3u30c7u30fcu30b7u30e7u30f3u30b7u30b9u30c6u30e0u306fMarengo 2.7u3092u4f7fu7528u3057u3066u30c6u30adu30b9u30c8u3092u57cbu3081u8fbcu307fu306bu5909u63dbu3057u3001u95a2u90a3u3059u308bu30d3u30c7u30aau306eu610fu5473u691cu7d22u3092u884cu3044u307eu3059u3002u30e6u30fcu30b6u30fcu304fu306fu3001u6c17u5206u3092u8abfu65e5u3057u305fu308au65b0u3057u3044u597du307fu3092u9078u629eu3059u308bu3053u3068u3067u3001u7d50u679cu3092u7d5eu308au8fbcu3080u3053u3068u304cu3067u304du307eu3059u3002u305du3057u3066u3001u30b3u30f3u30c6u30f3u30c4u30ecu30b3u30e1u30f3u30c9u30b7u30b9u30c6u30e0u306fQdrant Cloudu30b3u30ecu30afu30b7u30e7u30f3u304bu3089u66f4u65b0u3055u308cu305fu30ecu30b3u30e1u30f3u30c7u30fcu30b7u30e7u30f3u3092u53d6u5f97u3057u307eu3059u3002

u30a2u30d7u30eau30b1u30fcu30b7u30e7u30f3u304cu671fu5f85u3069u304au308au306bu52d5u4f5cu3059u308bu3088u3046u306bu3059u308bu306bu306fu3001u30b3u30ecu30afu30b7u30e7u30f3u306bu57cbu3081u8fbcu307fu30c7u30fcu30bfu3092u633fu5165u3059u308bu304bu3001u3053u306eu30a2u30d7u30eau30b1u30fcu30b7u30e7u30f3u3067u4f7fu7528u3055u308cu3066u3044u308bu3053u3061u308au306eu30b5u30f3u30d7u30ebu30c7u30fcu30bfu3092u53c2u7167u3057u3066u304fu3060u3055u3044u3002

u6e96u5099u624bu9806

1. TwelveLabs Playground u304bu3089 API u30adu30fcu3092u53d6u5f97u3057u3001u74b0u5883u5909u6570u3092u8a2du5b9au3057u307eu3059u3002

2. Github u304bu3089u30d7u30edu30b8u30a7u30afu30c8u3092u30afu30edu30fcu30f3u3057u307eu3059u3002

3. TwelveLabs u3068 Qdrant u306eu8a3cu660eu66f8u3092u542bu3080 .env u30d5u30a1u30a4u30ebu3092u4f5cu6210u3057u307eu3059u3002

4. Qdrant Cloud u30a3u30f3u30b9u30bfu30f3u30b9u3092u30bbu30c3u30c8u30a2u30c3u30d7u3057u307eu3059u3002Qdrant Cloud u30afu30e9u30b9u30bfu30fcu306eu30bbu30c3u30c8u30a2u30c3u30d7u30acu30a4u30c9u306bu5f93u3063u3066u304fu3060u3055u3044u3002

u3053u308cu3089u306eu30b9u30c6u30e3u30d7u304cu5b8cu4e86u3059u308bu3068u3001u958bu767au3092u59cbu3081u308bu6e96u5099u306fu6574u3044u307eu3059uff01

u30b3u30f3u30c6u30f3u30c4u30ecu30b3u30e1u30f3u30c9u30a2u30d7u30eau306eu30c1u30e5u30fcu30c8u30eau30a2u30eb

u3053u306eu30c1u30e5u30fcu30c8u30eau30a2u30ebu3067u306fu3001u30d3u30c7u30aau30b3u30f3u30c6u30f3u30c4u30ecu30b3u30e1u30f3u30c9u30a2u30d7u30eau30b1u30fcu30b7u30e7u30f3u3092u4f5cu6210u3059u308bu65b9u6cd5u3092u793au3057u307eu3059u3002u30d5u30edu30f3u30c8u30a8u30f3u30c9u306bu306f Next.jsu3001u30d0u30c3u30afu30a8u30f3u30c9u306bu306f CORS u3092u6709u52b9u305bu305a Flask API u3092u4f7fu7528u3057u307eu3059u3002u30b3u30f3u30c6u30f3u30c4u30ecu30b3u30e1u30f3u30c7u30fcu30b7u30e7u30f3u306eu30b3u30a2u306au30d0u30c3u30afu30a8u30f3u30c9u6a5fu80fdu306eu5b9fu88c5u3068u3001u30a2u30d7u30eau30b1u30fcu30b7u30e7u30f3u306eu30bbu30c3u30c8u30a2u30c3u30d7u306bu7126u70b9u3092u5f53u3066u307eu3059u3002

Qdrant Cloud u30afu30e9u30a4u30a2u30f3u30c8u306eu30bbu30c3u30c8u30a2u30c3u30d7u3001u57cbu3081u8fbcu307fu306eu751fu6210u3068u633fu5165u3001u30b3u30ecu30afu30b7u30e7u30f3u3068u306eu30a4u30f3u30bfu30e9u30afu30b7u30e7u30f3u306bu3088u308bu95a2u90a3u3059u308bu30d3u30c7u30aau306eu53d6u5f97u65b9u6cd5u3092u5b66u3073u307eu3059u3002u8a73u7d30u306au30b3u30fcu30c9u69cbu9020u3068u30bbu30c3u30c8u30a2u30c3u30d7u624bu9806u306bu3064u3044u306eu306fu3001GitHub u306e README.md u3092u3054u89a7u304fu3060u3055u3044u3002

1 - Qdrant u3078u306eu57cbu3081u8fbcu307fu306eu751fu6210u3068u633fu5165u306eu30acu30a4u30c9

u30b9u30c6u30c3u30d7 1 - u30bbu30c3u30c8u30a2u30c3u30d7u3068u4f9du5b58u95a2u4fc2

u307eu305au306fu5fc5u8981u306au4f9du5b58u95a2u4fc2u3092u30a4u30f3u30b9u30c8u30fcu30ebu3059u308bu3053u3068u304bu308au59cbu3081u307eu3057u30e7u3046u3002

u6b21u306bu3001u5fc5u8981u306au30e9u30a4u30d6u30e9u30eau3092u30a4u30f3u30ddu30fcu30c8u3057u307eu3059uff1a

import os
import uuid
import boto3
from botocore.exceptions import ClientError
import requests
from IPython.display import display, HTML
import shutil
import pandas as pd
from twelvelabs import TwelveLabs
from qdrant_client import QdrantClient, models
from qdrant_client.models import PointStruct
import time

u30b9u30c6u30c3u30d7 2 - u30b5u30fcu30d3u30b9u306eu8a2du5b9a

AWS S3u3001TwelveLabsu3001u304au3088u3073 Qdrant u3092u8a2du5b9au3057u307eu3059u3002u307eu305au3001u30d3u30c7u30aau30b9u30c8u30eau30fcu30dfu30f3u30b0u7528u306eu516cu958bu30d3u30c7u30aa URL u3092u751fu6210u3059u308bu305fu3081u306e AWS S3 u30d0u30b1u30c3u30c8u3092u30bbu30c3u30c8u30a2u30c3u30d7u3057u307eu3059u3002u305du306eu5f8cu3001u52b9u7387u7684u306au30d3u30c7u30aau57cbu3081u8fbcu307fu3068u691cu7d22u3092u5b9fu73feu3059u308bu305fu3081u306bu3001Qdrant u3068 TwelveLabs u30afu30e9u30a4u30a2u30f3u30c8u3092u521du671fu5316u3057u307eu3059u3002

# AWS S3 Configuration
AWS_ACCESS_KEY = "YOUR_AWS_ACCESS_KEY"
AWS_SECRET_KEY = "YOUR_AWS_SECRET_KEY"
AWS_BUCKET_NAME = "YOUR_BUCKET_NAME"
AWS_REGION = "us-east-1"  # Change to your region

# Initialize S3 client
s3_client = boto3.client(
    's3',
    aws_access_key_id=AWS_ACCESS_KEY,
    aws_secret_access_key=AWS_SECRET_KEY,
    region_name=AWS_REGION
)

# Twelve Labs Configuration
TWELVE_LABS_API_KEY = "YOUR_TWELVE_LABS_API_KEY"
twelvelabs_client = TwelveLabs(api_key=TWELVE_LABS_API_KEY)

# Qdrant Configuration
QDRANT_HOST = "YOUR_QDRANT_HOST"
QDRANT_API_KEY = "YOUR_QDRANT_API_KEY"
COLLECTION_NAME = "content_collection"
VECTOR_SIZE = 1024  # Size of embeddings from Twelve Labs

# Initialize Qdrant client
qdrant_client = QdrantClient(
    url=f"https://{QDRANT_HOST}",
    api_key=QDRANT_API_KEY,
    timeout=20,
    prefer_grpc=False
)

Qdrant u30afu30e9u30a4u30a2u30f3u30c8u3092u8a2du5b9au3059u308bu9645u306fu3001u7121u9650u306eu5f85u3061u3092u9632u304eu3001u9045u5ef6u3092u907fu3051u308bu305fu3081u306b timeout=20 u3092u8a2du5b9au3057u3066u304fu3060u3055u3044u3002Flask u306f gRPC u3067u306fu306au304f HTTP REST API u3092u30cdu30a4u30c6u30a3u30d6u306bu30b5u30ddu30fcu30c8u3057u3066u3044u308bu305fu3081u3001prefer_grpc=False u306bu8a2du5b9au3057u307eu3059u3002u3053u308cu306bu3088u308au3001u6a19u6e96u7684u306a HTTP u3092u4f7fu7528u3057u3066 Flask u3068 Qdrant u9593u306eu30b9u30e0u30fcu30bau306au901au4fe1u304cu4fddu8a3cu3055u308cu307eu3059u3002

u30d3u30c7u30aau30c7u30a3u30ecu30afu30c8u30eau3092u5b9au7fa9u3057u3001u305du306eu4e2du306e MP4 u30d5u30a9u30ebu30c3u30c0u30fcu306bu30a2u30afu30bbu30b9u3057u307eu3059u3002u3053u306eu30bbu30c3u30c8u30a2u30c3u30d7u306fu3001u305du306eu5f8cu306eu51e6u7406u30b9u30c6u30c3u30d7u306bu3068u3063u3066u4e0du53efu6b20u3067u3059u3002

# Get a list of video files
video_dir = "downloads/video_content"
video_files = [f for f in os.listdir(video_dir) if f.endswith('.mp4')]

u30b9u30c6u30c3u30d7 3 - AWS S3 u3078u306eu30d3u30c7u30aau306eu30a2u30c3u30d7u30edu30fcu30c9

u307eu305au3001u30d3u30c7u30aau3092 AWS S3 u30d0u30b1u30c3u30c8u306bu30a2u30c3u30d7u30edu30fcu30c9u3057u306eu30a2u30cau30a6u30f3u30b9 URL u3092u751fu6210u3059u308bu305fu3081u306eu95a2u6570u304fu304cu5fc5u8981u3067u3059u3002u516cu958b URL u304cu8fd4u3055u308cu3001u5404u30d3u30c7u30aau306eu30e1u30bfu30c7u30fcu30bfu3068u3057u3066u4fddu5b58u3055u308cu307eu3059u3002

def upload_to_s3(file_path, filename):
    try:
        # Upload the file
        s3_client.upload_file(
            file_path,
            AWS_BUCKET_NAME,
            f"videos-embed/{filename}",
            ExtraArgs={
                'ACL': 'public-read',
                'ContentType': 'video/mp4'
            }
        )

        # Generate the public URL
        url = f"https://{AWS_BUCKET_NAME}.s3.{AWS_REGION}.amazonaws.com/videos-embed/{filename}"
        print(f"Uploaded to S3: {url}")
        return url

    except ClientError as e:
        print(f"Error uploading to S3: {str(e)}")
        raise

u30b9u30c6u30c3u30d7 4 -  Marengo 2.7 u3092u4f7fu7528u3057u305fu30d3u30c7u30aau57cbu3081u8fbcu307fu306eu751fu6210

u6b21u306bu3001TwelveLabs u306e Marengo-retrieval-2.7 u30a8u30f3u30b8u30f3u3092u4f7fu7528u3057u3066u30d3u30c7u30aau57cbu3081u8fbcu307fu3092u751fu6210u3059u308bu95a2u6570u3092u4f5cu6210u3057u307eu3059u3002u30d3u30c7u30aau57cbu3081u8fbcu307fu306eu751fu6210u304au308fu308au633fu5165u306bu3064u3044u3066u306eu30b3u30fcu30c9u306fu3053u3061u308au3067u3054u89a7u3044u305fu3060u3051u307eu3059u3002

def create_video_embedding(video_path, max_retries=3, retry_delay=5):
    if not twelvelabs_client:
        raise ValueError("Twelve Labs API key not configured")

    retries = 0
    while retries < max_retries:
        try:
            print(f"Creating whole video embedding for {video_path}... (Attempt {retries+1}/{max_retries})")

            # Use video_embedding_scopes parameter set to ["clip", "video"] to get whole video embedding
            task = twelvelabs_client.embed.task.create(
                model_name="Marengo-retrieval-2.7",
                video_file=video_path,
                video_embedding_scopes=["clip", "video"]
            )

            print(f"Created task: id={task.id}, status={task.status}")
            task.wait_for_done(sleep_interval=3)
            task_result = twelvelabs_client.embed.task.retrieve(task.id)

            if task_result.status != 'ready':
                raise ValueError(f"Task failed with status: {task_result.status}")

            return task_result

        except Exception as e:
            print(f"Error creating embedding (attempt {retries+1}): {str(e)}")
            retries += 1
            if retries < max_retries:
                print(f"Retrying in {retry_delay} seconds...")
                time.sleep(retry_delay)
                retry_delay *= 2
            else:
                print("Max retries reached, giving up.")
                raise

video_embedding_scopes u306fu3001u30d3u30c7u30aau30afu30eau30c3u30d7u5168u4f53u306eu57cbu3081u8fbcu307fu3092u751fu6210u3059u308bu305fu3081u306bu3001u30afu30eau30c3u30d7u3068u30d3u30c7u30aau306eu4e21u65b9u306fu8a2du5b9au3055u308cu307eu3059u3002

u30b9u30c6u30c3u30d7 5 - Qdrant u3078u306eu30e1u30bfu30c7u30fcu30bfu4ed8u304du57cbu3081u8fbcu307fu306eu633fu5165

u751fu6210u3055u308cu305fu57cbu3081u8fbcu307fu3092 Qdrant u30d9u30afu30c8u30ebu30c7u30fcu30bfu30d9u30fcu30b9u306eu4fddu5b58u3059u308bu95a2u6570u3092u4f5cu6210u3057u307eu3059u3002u30d3u30c7u30aau57cbu3081u8fbcu307fu306fu3001u30e1u30bfu30c7u30fcu30bfu30dau30a4u30edu30fcu30c9u3092u4fbfu3044u30ddu30a4u30f3u30c8u3068u3057u3066u4fddu5b58u3055u308cu3001u691cu7d22u6027u304cu5411u4e0au3057u307eu3059u3002u30e1u30bfu30c7u30fcu30bfu5185u306eu30d3u30c7u30aa URL u306bu3088u308au3001u30b9u30c8u30eau30fcu30dfu30f3u30b0u304cu53efu80fdu306bu306au308au307eu3059u3002

def store_in_qdrant(task_result, video_id, s3_url, original_filename):
    if not qdrant_client:
        raise ValueError("Qdrant client not configured")

    try:
        print(f"Processing video embedding for {video_id}...")

        # The embedding will be in the segments with embedding_scope="video"
        if task_result.video_embedding and task_result.video_embedding.segments:
            video_segments = [s for s in task_result.video_embedding.segments
                             if hasattr(s, 'embedding_scope') and s.embedding_scope == 'video']

            if video_segments:
                print(f"Found video-scope embedding")
                embedding_vector = video_segments[0].embeddings_float
            else:
                # If no video scope segment is found, use the first segment as fallback
                print(f"No video-scope embedding found, using first available segment")
                embedding_vector = task_result.video_embedding.segments[0].embeddings_float
        else:
            raise ValueError("No embeddings found in the response")

        # Create a unique point structure for Qdrant storage
        point = PointStruct(
            id=uuid.uuid4().int & ((1<<64)-1), # Generate a unique 64-bit integer ID
            vector=embedding_vector, # Store the extracted embedding vector
            payload={
                'video_id': video_id,
                'video_url': s3_url,  # Store the public S3 URL of the video
                'is_url': True,
                'original_filename': original_filename # Save the original filename
            }
        )

	 # Insert the generated embedding point into the Qdrant collection
        qdrant_client.upsert(collection_name=COLLECTION_NAME, points=[point])
        print(f"Stored whole video embedding in Qdrant")
        return 1
    except Exception as e:
        print(f"Error storing in Qdrant: {str(e)}")
        raise

u30b9u30c6u30c3u30d7 6 - u30d3u30c7u30aau51e6u7406u30d1u30a4u30d7u30e9u30a4u30f3

u6b21u306bu3001u3059u3079u3066u306eu30b3u30f3u30ddu30fcu30cdu30f3u30c8u3092u63a5u7d9au3057u3001u30d1u30a4u30d7u30e9u30a4u30f3u3092u901au3058u3066u30d3u30c7u30aau3092u51e6u7406u3059u308bu30b9u30eau30e0u5316u3055u308cu305fu30d5u30edu30fcu3092u5b9au7fa9u3057u307eu3059u3002

u30d1u30a4u30d7u30e9u30a4u30f3u5168u4f53u3092u901au3058u3066u3001u30c7u30a3u30ecu30afu30c8u30eau5185u306eu3059u3079u3066u306eu30d3u30c7u30aau3092u51e6u7406u3057u307eu3059uff1a

1. AWS S3 u30d0u30b1u30c3u30c8u306bu30a2u30c3u30d7u30edu30fcu30c9

2. TwelveLabs u3092u4f7fu7528u3057u3066u57cbu3081u8fbcu307fu3092u751fu6210

3. Qdrant u306bu57cbu3081u8fbcu307fu3092u4fddu5b58

# Process each video
for filename in video_files[:5]:  # Process first 5 videos or you can setup as per convenience
    try:
        print(f"\nProcessing {filename}...")
        video_path = os.path.join(video_dir, filename)
        video_id = f"{str(uuid.uuid4())[:8]}_{filename}"
        
        # Upload to S3
        s3_url = upload_to_s3(video_path, video_id)
        
        # Generate embeddings
        task_result = create_video_embedding(video_path)
        
        # Store in Qdrant
        store_in_qdrant(task_result, video_id, s3_url, filename)
        
        print(f"Successfully processed {filename}")
    except Exception as e:
        print(f"Error processing {filename}: {str(e)}")

u3053u308cu3067u30d3u30c7u30aau306eu57cbu3081u8fbcu307fu3068u30e1u30bfu30c7u30fcu30bfu304c Qdrant Cloud u306bu4fddu5b58u3055u3082u3057u305fu3002u6b21u306eu30b9u30c6u30c3u30d7u306fu3001u53d6u5f97u306eu305fu3081u306bu30a2u30d7u30eau30b1u30fcu30b7u30e7u30f3u3092u30b3u30ecu30afu30b7u30e7u30f3u306bu63a5u7d9au3059u308bu3053u3068u3067u3059u3002

2 - Flask u3092u4f7fu7528u3057u305fu691cu7d22 API u306eu69cbu7bc9

u30b9u30c6u30c3u30d7 1 - CORS u30aau30eau30b8u30f3u306eu8a2du5b9a

u30afu30edu30b9u30aau30eau30b8u30f3u30eau30afu30a8u30b9u30c8u3092u6709u52b9u305bu305au3001Flask u30a2u30d7u30eau30b1u30fcu30b7u30e7u30f3u3067 CORSuff08Cross-Origin Resource Sharinguff09u3092u8a2du5b9au3059u308bu5fc5u8981u304cu3042u307eu3059u3002u3053u308cu306bu3088u308au3001u7570u306au308bu30c9u30e1u30a4u30f3u3067u30dbu30b9u30c8u3055u308cu3066u3044u308b Web u30afu30e9u30a4u30a2u30f3u30c8u306f API u306bu30a2u30afu30bbu30b9u3067u304du308bu3088u3046u306bu306au308au307eu3059u3002u30d0u30c3u30afu30a8u30f3u30c9u306eu5b8cu5168u306au5b9fu88c5u306f app.py u3067u3054u89a7u3044u305fu3060u3051u307eu3059u3002

app = Flask(__name__) 
CORS(app, resources={r"/*": {"origins": "*"}})

u30b9u30c6u30c3u30d7 2  - Qdrant u30b3u30ecu30afu30b7u30e7u30f3u306eu521du671fu5316

Qdrant u30b3u30ecu30afu30b7u30e7u30f3u304cu5b58u5728u3057u306au3044u5834u5408u306fu3001u521du671fu5316u3059u308bu5fc5u8981u304cu3042u307eu3059u3002u3053u306eu30bbu30c3u30c8u30a2u30c3u30d7u306bu3088u308au3001u30d9u30afu30c8u30ebu30c7u30fcu30bfu30d9u30fcu30b9u304cu30d3u30c7u30aau57cbu3081u8fbcu307fu3092u6b63u3057u304fu53d6u5f97u3067u304du308bu3088u3046u306bu306au308au307eu3059u3002TwelveLabs u30afu30e9u30a4u30a2u30f3u30c8u306fu3001u610fu5473u691cu7d22u6a5fu80fdu306eu305fu3081u306bu691cu7d22u30afu30a8u30eau306eu57cbu3081u8fbcu307fu3092u751fu6210u3059u308bu305fu3081u306bu521du671fu5316u3055u308cu307eu3059u3002

# Get credentials from environment variables
API_KEY = os.getenv('API_KEY')
QDRANT_HOST = os.getenv('QDRANT_HOST')
QDRANT_API_KEY = os.getenv('QDRANT_API_KEY')

# Qdrant Configuration
COLLECTION_NAME = "content_collection"
VECTOR_SIZE = 1024 # Dimension of vector embeddings

# Initialize clients
try:
    client = TwelveLabs(api_key=API_KEY)
    qdrant_client = QdrantClient(
        url=f"https://{QDRANT_HOST}",
        api_key=QDRANT_API_KEY,
        timeout=20
    )
    logger.info("Successfully initialized API clients")
except Exception as e:
    logger.error(f"Failed to initialize clients: {str(e)}")
    raise

def init_qdrant():
    try:
	 # Fetch all existing collections
        collections = qdrant_client.get_collections().collections
        collection_exists = any(col.name == COLLECTION_NAME for col in collections)
        if not collection_exists:
            # Create the collection with specified vector configuration if it doesn't exist
            qdrant_client.recreate_collection(
                collection_name=COLLECTION_NAME,
                vectors_config=VectorParams(
                    size=VECTOR_SIZE,
                    distance=Distance.COSINE # Use cosine similarity for retrieval
                )
            )
            logger.info(f"Created collection: {COLLECTION_NAME}")
    except Exception as e:
        logger.error(f"Qdrant initialization error: {str(e)}")
        raise

u30b9u30c6u30c3u30d7 3 - u30b8u30f3u30d7u30ebu306au691cu7d22u6a5fu80fdu306eu4f5cu6210

u3053u306eu30bbu30afu30b7u30e7u30f3u3067u306fu3001u30e6u30fcu30b6u30fcu304fu30d3u30c7u30aau3092u691cu7d22u3067u304du308bu3088u3046u306bu3059u308bu691cu7d22u30a8u30f3u30c9u30ddu30a4u30f3u30c8u3092u5b9fu88c5u3057u307eu3059u3002u3053u306eu30a8u30f3u30c9u30ddu30a4u30f3u30c8u306fu691cu7d22u30afu30a8u30eau3092u51e6u7406u3057u3001TwelveLabs u3092u4ecbu3057u3066u57cbu3081u8fbcu307fu3092u751fu6210u3057u3001Qdrant u304bu3089u985eu4f3cu3059u308bu30d9u30afu30c8u30ebu3092u53d6u5f97u3057u3001u4e00u81f4u3059u308bu7d50u679cu3092u8fd4u3057u307eu3059u3002

u30d3u30c7u30aau691cu7d22u7528u306eu691cu7d22u30a8u30f3u30c9u30ddu30a4u30f3u30c8u3092u4f5cu6210u3057u307eu3059u3002

@app.route('/search', methods=['POST'])
def search():
    # Ensure the request contains JSON data
    if not request.is_json:
        logger.warning("Missing JSON data")
        return jsonify({'error': 'Request must be JSON format'}), 400
        
    # Get and validate query
    data = request.get_json()
    query = data.get('query')
    if not query:
        logger.warning("Empty query parameter")
        return jsonify({'error': 'Missing query parameter'}), 400
    
    logger.info(f"Processing search: '{query}'")
    
    try:
        # Generate embedding for the search query
        formatted_query = f"Recommend - {query}"
        embedding_response = client.embed.create(
            model_name="Marengo-retrieval-2.7",
            text=formatted_query
        )

	 # Get the embedding vector
        vector = embedding_response.text_embedding.segments[0].embeddings_float
        
        # Similarity search from the Qdrant collection
        query_response = qdrant_client.query_points(
            collection_name=COLLECTION_NAME,
            query=vector,
            limit=10,
            with_payload=True
        )
        
        # Extract and format results
        search_results = query_response.points
        logger.info(f"Found {len(search_results)} matching results")
        
        # If no results, return empty list
        if not search_results:
            return jsonify([])
        
        # Build formatted response
        formatted_results = []
        for result in search_results:
            point_id = result.id
            score = float(result.score)
            payload = result.payload
            
            formatted_results.append({
                'video_id': payload.get('video_id', f"video_{point_id}"),
                'filename': payload.get('original_filename', payload.get('filename', 'video.mp4')),
                'start_time': float(payload.get('start_time', 0)),
                'end_time': float(payload.get('end_time', 30)),
                'score': score,
                'confidence': 'high' if score > 0.7 else 'medium',
                'url': payload.get('video_url')
            })
        
        logger.info(f"Returning {len(formatted_results)} results")
        return jsonify(formatted_results)
        
    except Exception as e:
        logger.exception(f"Search error: {str(e)}")
        return jsonify({'error': 'Search failed', 'details': str(e)}), 500

u3053u306eu3088u3046u306bu52d5u4f5cu3057u307eu3059uff1a

1. u30eau30afu30a8u30b9u30c8u304bu3089u691cu7d22u30afu30a8u30eau3092u53d6u5f97u3059u308b

2. TwelveLabs u3092u4f7fu7528u3057u3066u57cbu3081u8fbcu307fu30d9u30afu30c8u30ebu3092u751fu6210u3059u308b

3. Qdrant u3067u985eu4f3cu3059u308bu30d9u30afu30c8u30ebu3092u691cu7d22u3059u308b

4. u4e00u81f4u3059u308bu30d3u30c7u30aau3092u69cbu9020u5316u3055u308cu305fu5f62u5f0fu3067u8fd4u3059

u3053u306eu30c1u30e5u30fcu30c8u30eau30a2u30ebu30a2u30a4u30c7u30a2

u30a2u30d7u30eau30b1u30fcu30b7u30e7u30f3u306eu4ed5u7d44u307fu3092u7406u89e3u3059u308bu3053u3068u304cu3001u30e6u30fcu30b6u30fcu306eu30cbu30fcu30bau306bu5408u3063u305fu9769u65b0u306eu88fdu54c1u3092u4f5cu308bu3053u3068u306bu3064u306au304cu308au307eu3059u3002u30d3u30c7u30aau30b3u30f3u30c6u30f3u30c4u57cbu3081u8fbcu307fu306eu304au3059u3059u3081u306eu4f7fu3044u9053u306fu3053u3061u3081u3067u3059uff1a

ud83cudfaf u30d1u30fcu30bdu30cau30e9u30a4u30bau3055u308cu305fu5e83u544au633fu5165 u2014 u30d3u30c7u30aau306bu30b3u30f3u30c6u30adu30b9u30c8u306bu95a2u90a3u3059u308bu5e83u544au3092u52d5u7684u306bu633fu5165u3057u307eu3059u3002

u2699ufe0f u30eau30a2u30ebu30bfu30a4u30e0u985eu4f3cu5ea6u30deu30c3u30c1u30f3u30b0 u2014 u65b0u3057u3044u30b3u30f3u30c6u30f3u30c4u304fu30a2u30c3u30d7u30edu30fcu30c9u3055u308cu305fu3089u3001u77acu6642u306bu985eu4f3cu3059u308bu30d3u30c7u30aau3092u898bu3064u3051u307eu3059u3002

ud83dudcca u30c8u30ecu30f3u30c9u5206u6790 & u30a4u30f3u30b5u30a4u30c8 u2014 u57cbu3081u8fbcu307fu30d1u30bfu30fcu30f3u306eu57fau3065u3044u3066u3001u30d3u30c7u30aau30c8u30ecu30f3u30c9u3092u30afu30e9u30b9u30bfu30fcu5316u3057u3066u5206u6790u3057u307eu3059u3002

u304au308fu308au306b

u3053u306eu30c1u30e5u30fcu30c8u30eau30a2u30ebu3067u306fu3001u30d3u30c7u30aau7406u89e3u304cu3088u308au30b9u30deu30fcu30c8u3067u6b63u78bau306au30b3u30f3u30c6u30f3u30c4u30ecu30b3u30e1u30f3u30c7u30fcu30b7u30e7u30f3u3092u3069u306eu3088u3046u306bu751fu307fu51fau3059u304bu3092u793au3057u307eu3057u305fu3002u30d3u30c7u30aau57cbu3081u8fbcu307fu306b TwelveLabsu3001u9ad8u901fu306au30d9u30afu30c8u30ebu691cu7d22u306b Qdrant u3092u4f7fu7528u3059u308bu3053u3068u3067u3001u624bu52d5u306eu6587u5b57u8d77u3053u3057u3001u30bfu30b0u3001u30adu30fcu30efu30fcu30c9u3092u8d85u3048u3066u30d3u30c7u30aau30b3u30f3u30c6u30f3u30c4u3092u7406u89e3u3059u308bu30b7u30b9u30c6u30e0u3092u69cbu7bc9u3057u307eu3057u305fu3002u3053u306eu30a2u30d7u30edu30fcu30c1u306fu3001u3088u308au826fu3044u30ecu30b3u30e1u30f3u30c7u30fcu30b7u30e7u30f3u3092u63d0u4f9bu3057u3001u30e6u30fcu30b6u30fcu306eu95a2u5fc3u3092u5f15u304du3064u3051u3001u5927u898fu6a21u306au30d3u30c7u30aau30b3u30ecu30afu30b7u30e7u30f3u306bu3082u5bb9u6613u306bu30b9u30b1u30fcu30ebu3057u307eu3059u3002u30aau30fcu30d7u30f3u30bdu30fcu30b9u306eu30bdu30eau30e5u30fcu30b7u30e7u30f3u3068u3057u3066u3001u6559u80b2u304bu3089u30a8u30f3u30bfu30fcu30c6u30a4u30f3u30e1u30f3u30c8u306bu3044u305fu308bu307eu3067u3001u3055u307eu3056u307eu306au696du754cu306bu5408u308fu305bu3066u30abu30b9u30bfu30deu30a4u30bau3067u304du307eu3059u3002

u305du306eu4ed6u306eu30eau30bdu30fcu30b9

u57cbu3081u8fbcu307fu751fu6210u30a8u30f3u30b8u30f3u2014Marengo-retrieval-2.7 u306bu3064u3044u306eu8a73u7d30u3002TwelveLabs u3092u3055u3089u306bu63a2u6c42u3057u3001u30d3u30c7u30aau30b3u30f3u30c6u30f3u30c4u5206u30ebu306eu7406u89e3u3092u6df1u3081u308bu305fu3081u306bu3001u4ee5u4e0bu306eu30eau30bdu30fcu30beu3092u3054u78bau8a8du304fu306eu3060u3055u3044uff1a

• u30a4u30f3u30c6u30b0u30ecu30fcu30b7u30e7u30f3u3092u8a66u3059uff1a TwelveLabs Embed API Open Beta u306bu7b49u9332u3057u3066u3001u4ecau65e5u304bu3089 Qdrant u3092u4f7fu3063u305fu81eau5206u3060u3051u306e AI u30d3u30c7u30aau30a2u30d7u30eau30b1u30fcu30b7u30e7u30f3u306eu69cbu7bc9u3092u59cbu3081u307eu3057u30e7u3046u3002

• u4ed6u306eu30e6u30fcu30b9u30b1u30fcu30b9u3092u8abfu3079u308buff1a Qdrant Cloud QuickStart Guide u3092u53c2u7167u3057u3066u3001u30d3u30b8u30cdu30b9u306eu30cbu30fcu30bau306bu5408u308fu305bu305fu540cu69d8u306eu30efu30fcu30afu30d5u30edu30fcu3092u5b9fu88c5u3059u308bu65b9u6cd5u3092u5b66u3073u307eu3059u3002

• u8a08u8ad6u306bu53c2u52a0u3059u308buff1a u3053u306eu30a4u30f3u30c6u30b0u30ecu30fcu30b7u30e7u30f3u306bu304au3051u308bu30d5u30feedbacku3092 TwelveLabs Discord u3067u5171u6709u3057u3066u304fu3060u3055u3044u3002

• u30c1u30e5u30fcu30c8u30eau30a2u30ebu3092u63a2u3059uff1a u79c1u305fu3061u306eu7dcfu5408u7684u306au30c1u30e5u30fcu30c8u30eau30a2u30ebu3067 TwelveLabs u306eu6a5fu80fdu3092u3055u3089u306bu6df1u304fu63a2u3063u3066u307fu307eu3057u30e7u3046

u3053u308cu308au306eu30eau30bdu30fcu30b9u3092u4f7fu3063u3066u77e5u8b58u3092u5e83u3052u3001TwelveLabs u306eu30d3u30c7u30aau7406u89e3u6280u8853u3092u4f7fu7528u3057u305fu9769u65b0u306eu30a2u30d7u30eau30b1u30fcu30b7u30e7u30f3u3092u4f5cu6210u3059u308bu3053u3068u3092u304au52e7u3081u3057u307eu3059u3002