Infrastructure Systems: Digital Storytelling
REU Scholars: Nidhi Begur, Grace Gao, Emily Diaz-Silva, Juliette Plaisir, Dylan Heslop
REU Scholar Home Institutions: 91制片厂, CUNY Lehman
REU Mentor: Dr. Jason Hallstrom
CineSage: Building a Searchable Memory Pipeline for Long Video
View slide presentation for CineSage: Building a Searchable Memory Pipeline for Long VideoCities collect hours of CCTV footage every day, yet reviewing long recordings still requires significant time and human effort. Important events may appear only briefly or occur far apart, making them difficult to locate and connect. Although current artificial intelligence systems can analyze short clips, they often struggle to understand full-length videos or provide clear evidence for their answers. This research presents CineSage, a searchable video-memory system designed to support question answering over long-form footage. Rather than relying on a single description of the video, CineSage creates several types of memory, including scene summaries, frame-level descriptions, object and region details, dialogue and speaker information, and person Re-Identification across scenes. These memory sources are organized with timestamps so the system can retrieve relevant moments and use a large language model to produce an evidence-supported answer. Ablation testing was used to measure how much each memory source improved the system and to determine which combinations were most effective. CineSage was evaluated using a question benchmark created from multiple long-form movies, with verified answers and supporting video evidence reviewed by several language-model judges. The system performed strongly on general questions but had more difficulty with precise visual details and exact timestamps, with its best configurations achieving accuracy scores as high as 97%. Overall, CineSage demonstrates how combining multiple forms of searchable video memory can make long footage easier to review and support future applications in public safety and smart-city systems.