feat: AI Video summerization and subtitle generation with Lingo (Closes #1761) #1891
+3,318
−0
Conversation
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
📝 WalkthroughWalkthroughAdds a new Video Lingo AI demo under Changes
Sequence Diagram(s)sequenceDiagram
participant User as User (Browser)
participant Frontend as Vue Frontend
participant Backend as FastAPI
participant FFmpeg as FFmpeg
participant Whisper as Whisper Model
participant Groq as Groq API
participant Lingo as Lingo.dev API
User->>Frontend: Upload video + options (summarize, lang)
Frontend->>Backend: POST /process-video (multipart/form-data)
Backend->>FFmpeg: extract_audio(video_path)
FFmpeg-->>Backend: audio.wav
Backend->>Whisper: transcribe(audio.wav)
Whisper-->>Backend: segments & full text
alt summarize requested
Backend->>Groq: generate_text(full transcript)
Groq-->>Backend: summary
alt target language != en
Backend->>Lingo: translate_text(summary, target_lang)
Lingo-->>Backend: translated summary
end
Backend-->>Frontend: { summary }
else per-segment translation
alt target language != en
Backend->>Lingo: translate_text(each segment, target_lang)
Lingo-->>Backend: translated segments
end
Backend-->>Frontend: { segments: [{start,end,text}] }
end
Frontend->>User: Display summary and/or transcript with timings
Estimated code review effort🎯 4 (Complex) | ⏱️ ~60 minutes Suggested reviewers
Poem
🚥 Pre-merge checks | ✅ 3 | ❌ 2❌ Failed checks (1 warning, 1 inconclusive)
✅ Passed checks (3 passed)
✏️ Tip: You can configure your own custom pre-merge checks in the settings. Comment |
longkhan786
changed the title
longkhan786
changed the title
![coderabbitai[bot]](https://avatars.githubusercontent.com/in/347564?s=60&v=4){kind=small}
There was a problem hiding this comment.
Actionable comments posted: 17
🤖 Fix all issues with AI agents
In `@community/video-lingo-ai/.gitignore`:
- Around line 16-30: The .gitignore currently re-ignores the .vscode directory
after un-ignoring !.vscode/extensions.json, which nullifies the negation; update
the ignore rules so the negation takes effect by either removing the later
".vscode" entry or moving the negation "! .vscode/extensions.json" to the end of
the file; look for the ".vscode" and "!.vscode/extensions.json" entries and
adjust their order or remove the redundant ".vscode" line to ensure
extensions.json is tracked.
In `@community/video-lingo-ai/api/requirements.txt`:
- Around line 33-40: Update the vulnerable dependencies in requirements.txt:
remove the runtime pin for pip (it's build-time tooling and should not be in
runtime requirements), upgrade python-multipart to >=0.0.22 and setuptools to
>=78.1.1 to address CVE-2026-24486 and CVE-2024-6345/CVE-2025-47273
respectively, and for protobuf (protobuf==6.33.4) add a comment/placeholder to
monitor upstream PR `#25239` and avoid forcing an insecure workaround—do not
silently upgrade to an unpatched release; instead document the vulnerability and
revisit once a patched PyPI release is available.
In `@community/video-lingo-ai/api/src/main.py`:
- Around line 39-40: The variable returned as info from whisper.transcribe is
unused; rename it to _info to indicate intentional unused value and silence
linters—update the assignment in the call to whisper.transcribe (where segments,
info = whisper.transcribe(str(audio_file))) to segments, _info =
whisper.transcribe(str(audio_file)) and keep the subsequent segments =
list(segments) unchanged.
- Around line 33-35: The code uses file.filename directly which allows path
traversal; sanitize the name before creating video_path by extracting a safe
basename (e.g., Path(file.filename).name or os.path.basename) or replace with a
generated filename (e.g., uuid4 plus the original suffix) and then join with
UPLOAD_DIR; update the assignment to video_path and ensure all subsequent uses
reference that sanitized/generated name (refer to video_path, UPLOAD_DIR, and
file.filename).
- Around line 37-62: The handler currently extracts audio via
extract_audio(str(video_path)) and transcribes with whisper.transcribe, but
never removes the temporary video/audio files; wrap the transcription/processing
logic in a try/finally (or ensure cleanup after both the issummarize and normal
branches) and delete the temp audio_file and the uploaded video_path in the
finally block; reference the variables audio_file and video_path and ensure
cleanup occurs even if translate_text_with_lingo or generate_text throws so temp
files are removed on all paths.
- Around line 15-20: The CORS setup uses CORSMiddleware via app.add_middleware
with allow_origins=["*"] and allow_credentials=True which violates the Fetch
spec; change the configuration in the CORSMiddleware call to either set
allow_credentials=False or replace the wildcard allow_origins with an explicit
list of allowed origins (e.g., load from an environment variable or settings) so
credentialed requests get a specific Access-Control-Allow-Origin header; update
the parameters referenced in the CORSMiddleware call (allow_origins,
allow_credentials) accordingly to enforce the chosen behavior.
In `@community/video-lingo-ai/api/src/models.py`:
- Around line 12-14: Validate the GROQ_API_KEY environment variable before
constructing the Groq client: read os.getenv("GROQ_API_KEY") into a variable,
check it is non-empty, and if missing raise a clear exception (or log and exit)
so the failure happens at import time; then pass that validated key into
Groq(...) to create the client (referencing the client variable and Groq
constructor) and ensure generate_text() callers no longer rely on a late runtime
failure.
- Around line 28-51: The generate_text function directly indexes
response.choices[0] which can raise IndexError for empty or non-2xx Groq API
responses; update generate_text to validate the API response before accessing
response.choices (check response is not None, response.choices exists and
len(response.choices) > 0, and optionally verify response.status code or error
fields), and handle failure by logging/raising a clear exception or returning a
safe fallback error string so the FastAPI endpoint doesn't crash (use the same
function name generate_text and the local variable response to locate the
change).
In `@community/video-lingo-ai/api/src/utils/utils.py`:
- Around line 5-13: The extract_audio function currently uses a hardcoded
output_audio and ignores ffmpeg errors; update extract_audio to create a unique
temporary output file (e.g., via tempfile.NamedTemporaryFile or
tempfile.mkstemp) instead of "temp_audio.wav", run subprocess.run(command,
check=False) but capture the CompletedProcess returncode and stderr, verify
returncode == 0 and that the produced file exists and has nonzero size (or valid
WAV header) before returning; on failure, remove any stale temp file, raise a
RuntimeError including ffmpeg's stderr and returncode, and ensure the function
returns the unique temp path when successful.
- Around line 16-21: In translate_text_with_lingo, check that the environment
variable LINGODOTDEV_API_KEY is present before calling
LingoDotDevEngine.quick_translate; if it's missing, raise a clear ValueError (or
custom exception) explaining that LINGODOTDEV_API_KEY is required, and then pass
the validated api_key value to LingoDotDevEngine.quick_translate to avoid
passing None and producing a confusing SDK error.
In `@community/video-lingo-ai/i18n/index.js`:
- Around line 1-14: The project imports createI18n from "vue-i18n" (see the
createI18n import in i18n/index.js) but the package is missing from
package.json; add "vue-i18n" to the dependencies object in package.json (pick a
compatible version for your Vue setup, e.g., ^9.0.0 for Vue 3) and run yarn/npm
install so the import resolves at build/runtime.
In `@community/video-lingo-ai/README.md`:
- Around line 83-85: Update the fenced code block that currently contains just
the URL "http://localhost:5173" in README.md to include a language tag (e.g.,
add "text" after the opening triple backticks) so the block becomes a labeled
code fence and satisfies markdownlint rule MD040; locate the block by searching
for the literal URL inside the README and prepend the language tag to the
opening ``` fence.
- Line 11: The sentence "Due to limited resources please upload any english
spoked video." has grammar and capitalization issues; update the README sentence
to a polished version such as "Due to limited resources, please upload any
English-spoken video." (capitalize "English", add a comma after the introductory
clause, and fix "spoked" to "spoken") so replace the existing line accordingly
in the README content.
In `@community/video-lingo-ai/src/App.vue`:
- Around line 48-49: The request is manually setting headers: { "Content-Type":
"multipart/form-data" } which prevents the browser/axios from appending the
multipart boundary; remove that manual Content-Type header in the request (leave
headers undefined or remove only the Content-Type key) so that FormData is sent
correctly (the code that builds the request around params: { issummarize:
isSummarize.value, lang: lang.value } and the FormData payload will then allow
axios/browser to set the correct multipart/form-data; boundary automatically).
In `@community/video-lingo-ai/src/components/VideoUpload.vue`:
- Around line 190-203: The template iterates result.segments directly causing
seg.start/seg.end to be undefined when segments is missing; change to iterate a
normalized array (e.g., a computed/props property like resultSegments) that
returns Array.isArray(result.segments) ? result.segments : [] (or wraps a single
transcript into an array) and update the v-for to use that normalized name,
ensuring formatTime is only called with valid numbers; alternatively add a
template guard like v-if="Array.isArray(result.segments) &&
result.segments.length" around the transcript-list to prevent rendering when
segments isn't an array.
- Around line 45-46: The request config in VideoUpload.vue currently sets
headers: { "Content-Type": "multipart/form-data" } which strips the required
multipart boundary; remove that manual header from the Axios request (leave
params: { issummarize: isSummarize.value } intact) so Axios can automatically
set Content-Type with the correct boundary when sending the FormData. Locate the
Axios call/config in VideoUpload.vue where headers is defined and delete the
Content-Type entry or the whole headers object for that request.
In `@community/video-lingo-ai/src/composables/useI18n.js`:
- Around line 1-22: The t function in useI18n reads messages[lang.value] which
can be undefined for unsupported locales and only en/hi are loaded; update
messages to include all available locale bundles (e.g., add imports for other
locales) and make t robust by checking messages[lang.value] exists before
indexing—fallback to messages.en or return key; also validate setLang to only
assign known keys (check against Object.keys(messages)) and ignore or fallback
when an unsupported locale is passed.
🧹 Nitpick comments (2)
community/video-lingo-ai/src/App.vue (1)
44-46: Make the API base URL configurable.
Line 45 hardcodeshttp://127.0.0.1:8000/process-video, which prevents deployment outside local development. Use Vite environment variables for portability.🛠️ Suggested change
import { ref } from "vue" import axios from "axios" import { useI18n } from "./composables/useI18n" +const API_BASE_URL = import.meta.env.VITE_API_BASE_URL ?? "http://127.0.0.1:8000" + const { t, lang, setLang } = useI18n()- "http://127.0.0.1:8000/process-video", + `${API_BASE_URL}/process-video`,community/video-lingo-ai/src/components/VideoUpload.vue (1)
41-42: Extract hardcoded API URL to environment variable in both VideoUpload.vue and App.vue.The API base URL
http://127.0.0.1:8000is hardcoded in two locations (VideoUpload.vue line 42 and App.vue line 45), making deployments outside local development impossible. Configure this via Vite's environment variables for portability.🛠️ Suggested approach
Add to vite.config.js or create a
.envfile with:VITE_API_BASE_URL=http://127.0.0.1:8000Then update both files to use:
+const API_BASE_URL = import.meta.env.VITE_API_BASE_URL ?? "http://127.0.0.1:8000" + - "http://127.0.0.1:8000/process-video", + `${API_BASE_URL}/process-video`,
longkhan786
changed the title
There was a problem hiding this comment.
Actionable comments posted: 4
🤖 Fix all issues with AI agents
In `@community/video-lingo-ai/README.md`:
- Line 61: Update the README instruction "Create a `.env` file" to explicitly
state that the `.env` file must be created inside the api directory; modify the
sentence to read something like "Create a `.env` file in the api directory" so
readers know the correct location, and ensure any subsequent steps that
reference environment variables assume that same api/.env placement.
- Line 89: Replace the inaccurate language claim in the README by updating the
sentence that currently reads "View the transcript and summary in
English/Hindi." to list all supported locales from the i18n config (English,
Spanish, French, and Hindi) and optionally include locale codes (en, es, fr,
hi); locate the exact string in README.md and update it to something like "View
the transcript and summary in English, Spanish, French, and Hindi (en, es, fr,
hi)" so the docs match the i18n files.
- Around line 40-42: Replace the placeholder "<your-repo-url>" in the bash
snippet with the actual repository clone URL and correct the directory change
command from "cd video-lingo-ai/api" to "cd community/video-lingo-ai" so the
instructions reflect the real repo and path; update the README.md bash block
that contains the git clone and cd commands accordingly to use the real URL and
the corrected path.
- Around line 76-79: Update the README instructions so users switch into the
frontend directory before running the npm commands shown in the code block (the
lines containing "npm install" and "npm run dev"); modify the steps to include a
"cd" into the appropriate frontend folder (e.g., "cd frontend" or "cd .."
depending on repo layout) immediately before those two commands so it's clear
where to run them.
🧹 Nitpick comments (1)
community/video-lingo-ai/README.md (1)
107-107: Consider removing or rephrasing the informal acknowledgment.The sentence "Thank you for Organizing this" has inappropriate capitalization and feels out of place in technical documentation. Consider removing it or moving acknowledgments to a separate CONTRIBUTORS or ACKNOWLEDGMENTS section.
✏️ Optional refinement
-Thank you for Organizing this +---Or if you want to keep an acknowledgment:
-Thank you for Organizing this +Special thanks to the Lingo.dev team for organizing the community challenge.
There was a problem hiding this comment.
Actionable comments posted: 2
🤖 Fix all issues with AI agents
In `@community/video-lingo-ai/api/requirements.txt`:
- Line 11: In requirements.txt, the package entry "dotenv==0.9.9" is invalid on
PyPI; replace that token with the correct package name "python-dotenv==0.9.9" or
simply remove the "dotenv==0.9.9" line since "python-dotenv" is already listed
elsewhere — update the requirements.txt entry to use "python-dotenv" (or remove
the duplicate) so pip install succeeds.
In `@community/video-lingo-ai/README.md`:
- Line 64: Update the README example and Tech Stack text to reference Groq
instead of OpenAI: replace the environment variable example key OPENAI_API_KEY
with GROQ_API_KEY and update the Tech Stack line that currently says "OpenAI API
(for summarization & translation)" to mention Groq API (or Groq SDK) for
summarization & translation so it matches the backend usage and requirements
(groq==1.0.0).
🧹 Nitpick comments (2)
community/video-lingo-ai/.gitignore (1)
26-26:.envignore may conflict with Python virtual environment directory naming.The pattern
.envwill ignore both the.envfile (environment variables) and any directory named.env. This is typically fine, but ensure clarity—if you specifically want to ignore the env file, consider renaming it or using a more specific pattern like*.envor.env.*for additional env files.community/video-lingo-ai/src/components/VideoUpload.vue (1)
31-33: Hardcoded strings instead of i18n.Unlike
App.vuewhich usest("selectVideoAlert"), this component uses hardcoded English strings (e.g.,"Please select a video","Upload failed"). Consider integrating theuseI18ncomposable for consistency.
Contributor
|
Hi @longkhan786, some of the AI comments are still unaddressed. Please push new commits after addressing them. Thank you for your awesome work! ❤️ |
Author
all comments are fixed now |
Contributor
|
@coderabbitai full review |
✅ Actions performedFull review triggered. |
There was a problem hiding this comment.
Actionable comments posted: 1
🤖 Fix all issues with AI agents
In `@community/video-lingo-ai/package.json`:
- Around line 11-14: package.json is missing the vue-i18n dependency used by
i18n/index.js (which imports createI18n); update the dependencies object in
package.json to include "vue-i18n" with a 9.x version (e.g. "vue-i18n":
"^9.x.x") so the import in i18n/index.js resolves and installs correctly.
Comment on lines
+11
to
+14
| "dependencies": { | ||
| "axios": "^1.13.2", | ||
| "vue": "^3.5.24" | ||
| }, |
There was a problem hiding this comment.
🧩 Analysis chain
🏁 Script executed:
#!/bin/bash
# Check if vue-i18n is imported in the codebase
rg -n "vue-i18n" community/video-lingo-ai/
# Check the i18n index.js for imports
if [ -f "community/video-lingo-ai/i18n/index.js" ]; then
echo "=== Contents of i18n/index.js ==="
cat -n community/video-lingo-ai/i18n/index.js
else
echo "i18n/index.js not found"
fi
# Check package.json to confirm current dependencies
echo "=== Current dependencies in package.json ==="
cat community/video-lingo-ai/package.jsonRepository: lingodotdev/lingo.dev
Length of output: 948
Add vue-i18n to dependencies.
The i18n setup at i18n/index.js imports createI18n from "vue-i18n", but this package is not listed in package.json dependencies. This will cause installation and runtime failures. Add "vue-i18n": "^9.x.x" to the dependencies object.
🤖 Prompt for AI Agents
In `@community/video-lingo-ai/package.json` around lines 11 - 14, package.json is
missing the vue-i18n dependency used by i18n/index.js (which imports
createI18n); update the dependencies object in package.json to include
"vue-i18n" with a 9.x version (e.g. "vue-i18n": "^9.x.x") so the import in
i18n/index.js resolves and installs correctly.
Contributor
There was a problem hiding this comment.
⚠️ Potential issue | 🔴 Critical🧩 Analysis chain
🏁 Script executed:#!/bin/bash # Check if vue-i18n is imported in the codebase rg -n "vue-i18n" community/video-lingo-ai/ # Check the i18n index.js for imports if [ -f "community/video-lingo-ai/i18n/index.js" ]; then echo "=== Contents of i18n/index.js ===" cat -n community/video-lingo-ai/i18n/index.js else echo "i18n/index.js not found" fi # Check package.json to confirm current dependencies echo "=== Current dependencies in package.json ===" cat community/video-lingo-ai/package.jsonRepository: lingodotdev/lingo.dev
Length of output: 948
Add
vue-i18nto dependencies.The i18n setup at
i18n/index.jsimportscreateI18nfrom"vue-i18n", but this package is not listed inpackage.jsondependencies. This will cause installation and runtime failures. Add"vue-i18n": "^9.x.x"to the dependencies object.🤖 Prompt for AI Agents
@longkhan786 please fix
There was a problem hiding this comment.
Seems like the humans are having a chat. I'll hop back into my burrow for now. If you need me again, just tag @coderabbitai in a new comment, and I'll come hopping out!
Author
There was a problem hiding this comment.
⚠️ Potential issue | 🔴 Critical
🧩 Analysis chain
🏁 Script executed:#!/bin/bash # Check if vue-i18n is imported in the codebase rg -n "vue-i18n" community/video-lingo-ai/ # Check the i18n index.js for imports if [ -f "community/video-lingo-ai/i18n/index.js" ]; then echo "=== Contents of i18n/index.js ===" cat -n community/video-lingo-ai/i18n/index.js else echo "i18n/index.js not found" fi # Check package.json to confirm current dependencies echo "=== Current dependencies in package.json ===" cat community/video-lingo-ai/package.jsonRepository: lingodotdev/lingo.dev
Length of output: 948
Addvue-i18nto dependencies.
The i18n setup ati18n/index.jsimportscreateI18nfrom"vue-i18n", but this package is not listed inpackage.jsondependencies. This will cause installation and runtime failures. Add"vue-i18n": "^9.x.x"to the dependencies object.
🤖 Prompt for AI Agents@longkhan786 please fix
I have added "vue-i18n": "^9.14.5"
this issue is fixed now
sumitsaurabh927
changed the title
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
2 participants
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Summary
Overview
Video Lingo AI is a web application that allows users to:
This tool is built to demonstrate how AI can help automatically understand, summarize, and localize video content.
Closes #1761
Summary by CodeRabbit
New Features
Documentation
Chores
✏️ Tip: You can customize this high-level summary in your review settings.