🎉 Join the community of fellow Multimodal Minds today!

Unleash the Power of Auto-Generating Video Title, Topics, and Hashtags

Author

Meeran Kim

Date Published

March 22, 2024

Introduction

In continuation of my previous application and post, I am introducing a new tool designed to assist influencers in efficiently preparing for video uploads on platforms like YouTube. After completing video creation, influencers often face the challenge of crafting an engaging title and relevant hashtags. This application leverages the Twelve Labs API to streamline this process.

Rather than starting from scratch, users simply upload or index their video and click 'Generate' to receive accurate title suggestions, topic recommendations, and hashtag ideas within seconds. By eliminating the need to explain the video content to the AI, this tool empowers influencers to quickly access tailored suggestions, enhancing their upload workflow and engagement potential.

Let's delve into the step-by-step process of building this application.

Prerequisites

You should have your Twelve Labs API Key. If you don’t have one, visit the Twelve Labs Playground, sign up, and generate your API key.
The repository containing all the files for this app is available on Github.
(Good to Have) Basic knowledge in JavaScript, Node, React, and React Query. Don't worry if you're not familiar with these though. The key takeaway from this post will be to see how this app utilizes the Twelve Labs API!

‍

How the App is Structured

The app consists of five main components; GenerateTitlesAndHashtags, VideoFileUploadForm, Video, InputForm, and Result.

GenerateTitlesAndHashtags: It serves as the parent container for the other components. It holds key states that are shared with its descendants.
VideoUrlUploadForm: It is a simple form that receives the video file and index the video using TwelveLabs API. It also shows the video that is in the process of indexing as well as the status of an indexing task until the indexing is complete.
Video: It shows a video by a given url. It is reused within two different components.
InputForm: It is a form consisting of three check boxes; Topic, Title and Hashtags. A user can check or uncheck each field.
Result: It shows the result of the fields checked from the inputForm by calling the TwelveLabs API (‘/gist’ endpoint)

The app also has a server that stores all the code involving the API calls and the apiHooks.js which is a set of custom React Query hooks for managing state, cache, and fetching data.

Now, let’s take a look at how these components work along with the Twelve Labs API.

‍

How the App Works with Twelve Labs API

‍

1 - Showing the Most Recent Video of an Index

This app only works with one video, which is the most recently uploaded video of an index. Thus, on mount, the app shows the most recent video of a given index by default. Below is the process of how it works.

Get all videos of a given index in the App.js (GET Videos)
Extract the first video’s id from the response and pass it down to GenerateTitlesAndHashtags.js
Get details of a video using the video id, extract the video HLS (HTTP Live Streaming) url and pass it down to Video.js (GET Video)

So we make two GET requests to the Twelve Labs API in this flow of getting the videos and showing the first video on the page. Let’s take each step in detail.

‍

1.1 - Get all videos of a given index in the App.js

Inside the app, the videos are returned by calling the react query hook useGetVideos which makes the request to the server. The server then makes a GET request to the Twelve Labs API to get all videos of an index. (💡Find details in the API document - GET Videos)

‍Server.js (line 53 - 73)

/** Get videos */

app.get("/indexes/:indexId/videos", async (request, response, next) => {
 const params = {
   page_limit: request.query.page_limit,
 };


 try {
   const options = {
     method: "GET",
     url: `${API_BASE_URL}/indexes/${request.params.indexId}/videos`,
     headers: { ...HEADERS },
     data: { params },
   };
   const apiResponse = await axios.request(options);
   response.json(apiResponse.data);
 } catch (error) {
   const status = error.response?.status || 500;
   const message = error.response?.data?.message || "Error Getting Videos";
   return next({ status, message });
 }
});

The returned data (videos) looks like below.


{
	"data": [
		{
			"_id": "64eadb7e37db8fa679cb47e2",
			"created_at": "2024-03-08T09:34:06Z",
			"metadata": {
				"duration": 213.72,
				"engine_ids": [
					"pegasus1",
				],
				"filename": "myFile.mp4",
				"fps": 25,
				"height": 720,
				"size": 25014061,
				"width": 1280
			}
		},


		{ 2nd video },
		{ 3rd video }, 
		…
}

‍

1.2 - Extract the first video’s id from the response and pass it down to SummarizeVideo.js

Based on the returned videos, we’re passing down the first video’s id to the GenerateTitlesAndHashtags.js component.

‍App.js (line 34 - 38)


           <GenerateTitlesAndHashtags
             index={apiConfig.INDEX_ID}
             videoId={videos.data[0]?._id || null} //passing down the id
             refetchVideos={refetchVideos}
           />

‍

1.3 - Get details of a video using the video id, extract the video source url and pass it down to Video.js

Similar to what we’ve seen from the previous step in getting videos, to get details of a video, we’re using the react query hook useGetVideo which makes the request to the server. The server then makes a GET request to the Twelve Labs API to get details of a specific video. (💡Find details in the API document - GET Video)

‍Server.js (line 76 - 97)

/** Get a video of an index */

app.get(
 "/indexes/:indexId/videos/:videoId",
 async (request, response, next) => {
   const indexId = request.params.indexId;
   const videoId = request.params.videoId;


   try {
     const options = {
       method: "GET",
       url: `${API_BASE_URL}/indexes/${indexId}/videos/${videoId}`,
       headers: { ...HEADERS },
     };
     const apiResponse = await axios.request(options);
     response.json(apiResponse.data);
   } catch (error) {
     const status = error.response?.status || 500;
     const message = error.response?.data?.message || "Error Getting a Video";
     return next({ status, message });
   }
 }
);

It returns the details of a video including what we need, the url of a video! You might have noticed that the video url was not available from GET Videos. As a reminder, this is why we’re making the GET Video request here.

One more note here is that as this app only allows a user to upload/index a video file from a local device, the video url will be inside the “hls” object. (while in my previous app, as the app supported uploading by youtube url, the youtube url was available inside the “source” object)


{
	"_id": "64eadb7e37db8fa679cb47e2",
	"created_at": "2024-03-08T09:34:06Z",
	"indexed_at": "2024-03-08T09:39:14Z",
	"metadata": {
		"duration": 213.72,
		"engine_ids": [
			"pegasus1",
		],
		"filename": "myFile.mp4",
		"fps": 25,
		"height": 720,
		"size": 25014061,
		"video_title": "myFile.mp4",
		"width": 1280
	},
	"hls": {
		"video_url": "https://deuqpmn4rs7j5.cloudfront.net/…",
		"thumbnail_urls": [
			"https://deuqpmn4rs7j5.cloudfront.net/…"
		],
		"status": "COMPLETE",
		"updated_at": "2024-03-08T09:35:21.813Z"
	}
}

Based on the returned data (video), we’re passing down its url to the Video component where the React Player is rendered using the url.

GenerateTitlesAndHashtags.js (line 105 - 111)


 {video && (
             <Video
               url={video.hls?.video_url} // passing down the url
               width={"381px"}
               height={"214px"}
             />
           )}

‍

2 - Uploading/Indexing a Video by Video File

In this app, a user can upload and index a video by selecting a video file from the local device. We’re assuming that a main user of this application would be an influencer who has a video ready to be published but not yet publicly available.

Once you submit a video indexing request (we call it a ‘task’), then we can receive the progress of the indexing task. I also made a video visible during the indexing process so that a user can confirm and watch the video while the indexing is in progress. Please note that the video url is not instantly available from the beginning of the indexing process; it becomes available about at the middle of the indexing process.

Create a video indexing task by a video file in VideoUrlUploadForm.js (POST Task)
Receive and show the progress of the indexing task in Task.js (GET Task)

Let’s take a look at each step one by one.

‍

2.1 - Create a video indexing task by a video file in VideoUrlUploadForm.js

When a user selects a video file from his/her local device and submits the videoFileUploadForm, the video indexing process starts.

indexYouTubeVIdeo puts together the necessary data that needs to be sent for making the API request (e.g., language, index id, video file) as a form and makes a POST request to the server. The server then makes a post request to Twelve Labs API’s ‘/tasks’ endpoint. (💡Find details in the API document - POST Task)

server.js (line 119 - 158)

/** Index a Youtube video for analysis, returning a task ID */

app.post(
 "/index",
 upload.single("video_file"),
 async (request, response, next) => {
   const formData = new FormData();


   // Append data from request.body
   Object.entries(request.body).forEach(([key, value]) => {
     formData.append(key, value);
   });


   const blob = new Blob([request.file.buffer], {
     type: request.file.mimetype,
   });


   formData.append("video_file", blob, request.file.originalname);


   const options = {
     method: "POST",
     url: `${API_BASE_URL}/tasks`,
     headers: {
       "x-api-key": TWELVE_LABS_API_KEY,
       accept: "application/json",
       "Content-Type":
         "multipart/form-data; boundary=---011000010111000001101001",
     },
     data: formData,
   };
   try {
     const apiResponse = await axios.request(options);
     response.json(apiResponse.data);
   } catch (error) {
     const status = error.response?.status || 500;
     const message =
       error.response?.data?.message || "Error indexing a YouTube Video";
     return next({ status, message });
   }
 }
);

It returns an id of the video task that has just been created.


{
	"_id": "65e9f732bb29f13bdd6f305a"
}

‍

2.2 - Receive and show the progress of the indexing task in Task.js

With the returned task id from the previous step of getting details of a video, we will use this task id to get details of the task and keep updating the task status to the user.

So I’ve set the Task component to render inside the VideoFileUploadForm when there is a task id.

‍VideoFileUploadForm.js (line 144 - 149)


{taskId && (
               <Task
                 taskId={taskId}
                 refetchVideos={refetchVideos}
               />
             )}

Inside the Task component, we’re receiving the data by using the useGetTask React Query hook which makes a GET request to the server. The server then makes a GET request to the Twelve Labs API as below. (💡Find details in the API document - GET Task)

server.js (line 160 - 177)

/** Check the status of a specific indexing task */
app.get("/tasks/:taskId", async (request, response, next) => {
 const taskId = request.params.taskId;


 try {
   const options = {
     method: "GET",
     url: `${API_BASE_URL}/tasks/${taskId}`,
     headers: { ...HEADERS },
   };
   const apiResponse = await axios.request(options);
   response.json(apiResponse.data);
 } catch (error) {
   const status = error.response?.status || 500;
   const message = error.response?.data?.message || "Error getting a task";
   return next({ status, message });
 }
});

It returns the task details as below.


{
	"_id": "65e9f732bb29f13bdd6f305a",
	"index_id": "65e9f6d6bb29f13bdd6f3059",
	"video_id": "65e9f73248db9fa679cb47a2",
	"status": "indexing",
	"metadata": {
		"filename": "fileName.mp4",
		"duration": 327.326988,
		"width": 1920,
		"height": 1080
	},
	"created_at": "2024-03-07T17:19:46.277Z",
	"updated_at": "2024-03-07T17:27:27.135Z",
	"estimated_time": "2024-03-07T17:25:59.621Z",
	"type": "index_task_info",
	"hls": {
		"video_url": "...",
		"thumbnail_urls": [
			"..."
		],
		"status": "COMPLETE",
		"updated_at": "2024-03-07T17:21:23.921Z"
	}
}

Unless the “status” is “ready”, the useGetTask hook will refetch the data every 5,000 ms so that a user can see the progress of the task in real-time. Check how I leveraged the refetchInterval property of useQuery below.

apiHooks.js (line 70 - 84)


export function useGetTask(taskId) {
 return useQuery({
   queryKey: [keys.TASK, taskId],
   queryFn: () =>
     apiConfig.SERVER.get(`${apiConfig.TASKS_URL}/${taskId}`).then(
       (res) => res.data
     ),
   refetchInterval: (data) => {
     return data?.status === "ready" || data?.status === "failed"
       ? false
       : 5000;
   },
   refetchIntervalInBackground: true,
 });
}

‍

3 - Receiving User Inputs and Generate/Show Results

Now we’re at the core and fun part - generating topic, title, and hashtags of a video! We’re receiving the user input then using the Twelve Labs API’s gist endpoint to generate the written topic, title, and hashtags of a video.

Receive a user input from the checkbox form in InputForm.js
Based on the user input on each field prompt, make ‘/gist’ API call in Result.js (POST Titles, topics, or hashtags)
Show the results in Result.js

Let’s dive into each step.

‍

3.1 - Receive a user input from the checkbox form in InputForm.js

InputForm is a simple form consisting of the three checkbox fields; Topic, Title, and Hashtags. Whenever a user checks or unchecks each field, each field name is added to or deleted from the ‘types’ state which is a set of types. This is because the ‘types’ property is required to make the request to Twelve Labs API’s ‘/gist’ endpoint in the next step.

I’ve set the field1, field2, field3 names in the GenerateTitlesAndHashtags.js.

GenerateTitlesAndHashtags.js (line 28)


 const [field1, field2, field3] = ["topic", "title", "hashtag"];


InputForm.js (line 31 - 47) 
if (topicRef.current?.checked) {
     types.add(field1);
   } else {
     types.delete(field1);
   }


   if (titleRef.current?.checked) {
     types.add(field2);
   } else {
     types.delete(field2);
   }


   if (hashtagRef.current?.checked) {
     types.add(field3);
   } else {
     types.delete(field3);
   }

‍

3.2 - Based on the user input on each field prompt, make ‘/gist’ API call in Result.js

When the form has been submitted and the video id and types are set, useGenerateTitleTopicHashtag hook will be called from the Result.js. The hook will then make the request to the server where the API request to Twelve Labs API is made. (💡Find details in the API document - POST Titles, topics, or hashtags)

Just so you know, we can generate title, topic, and hashtags by a single API request while we had to make 3 separate requests for each of the summary, chapters, and highlights in the previous application (See 3-2). Find details on the ‘/summary’ endpoint here.

‍server.js (line 98 - 117)

/** Generate gist of a video */
app.post("/videos/:videoId/gist", async (request, response, next) => {
 const videoId = request.params.videoId;
 let types = request.body.data;
 try {
   const options = {
     method: "POST",
     url: `${API_BASE_URL}/gist`,
     headers: { ...HEADERS, accept: "application/json" },
     data: { ...types, video_id: videoId },
   };
   const apiResponse = await axios.request(options);
   response.json(apiResponse.data);
 } catch (error) {
   const status = error.response?.status || 500;
   const message =
     error.response?.data?.message || "Error Generating Gist of a Video";
   return next({ status, message });
 }
});

It returns an object consisting of response “id” and the corresponding types that you included in your request. For example, if you included all three types in your request, you will see the result data including all three types; topic, title, and hashtags like below.


{
	"id": "c7a420ef-ef2a-4a68-af65-3078af2d92fc",
	"title": "One-Pot Chicken and Rice: Cooking a Flavorful Dish from Marination to Serving",
	"topics": [
		" One-Pot Chicken and Rice Cooking Tutorial"
	],
	"hashtags": [
		"OnePotMeals",
		"ChickenRecipes",
		"RiceCooking",
		"MarinationTechniques",
		"CookingTutorials",
		"Spices",
		"YogurtMarinade"
	]
}

‍

3.3 - Shows the results in Result.js

Based on the response we get from the above step, the results are shown in the Result component. Note that “topics” and “hashtags” are in the format of an array, while the “title” is a string.

‍

Conclusion

With the Twelve Labs' “/gist” API, you can easily generate the title, topics, and hashtags of any video. It can be your best friend when you've already created a video and now need to come up with an engaging yet accurate title and hashtags before publishing it on platforms like YouTube. This tool can serve as your title/topics/hashtags generator, highly specialized in videos. I hope this app and tutorial were helpful in providing ideas for you. As always, Happy Coding!

‍

What's Next?

Check out the Quickstart tutorial, and begin building amazing apps with Twelve Labs.
Try out our Playground. Default video credits are 10 hours.
Follow us on X (Twitter) and LinkedIn.
Join our Discord community to connect with fellow users and developers.