An Electron-based desktop application that creates a searchable visual memory of your digital life by continuously capturing screen activity and transforming it into queryable, time-aware intelligence powered by Tavily API.

Digital Memory acts as your "digital photographic memory" - capturing, indexing, and enriching your screen activity to make everything you've seen searchable. Built for the Tavily Multimodal Web Agents Hackathon, it demonstrates innovative use of Tavily's APIs to bridge local and web knowledge.

• Node.js: v18.0.0 or higher (v23.9.0 recommended)
• npm: v8.0.0 or higher
• Operating System: macOS, Windows, or Linux
• RAM: 4GB minimum, 8GB recommended
• Storage: 2GB free space for app + captured data

• Tavily API Key: Get yours at tavily.com
  • Free tier: 1,000 requests/month
  • Hackathon tier: Enhanced limits available

# Clone the repository
git clone [repository-url]
cd digital-memory

# Install dependencies
npm install

# Install additional type definitions if needed
npm install --save-dev @types/better-sqlite3

The app works out of the box without configuration. For Tavily API integration (when implemented), create a .env file:

# Tavily API Integration (placeholder - not yet implemented)
TAVILY_API_KEY=tvly-YOUR_API_KEY_HERE

Note: Environment variables are not currently used by the application. The capture interval is dynamically adjusted based on user activity (15-60 seconds).

# Start in development mode with hot reload
npm start

# Run TypeScript compiler check
npx tsc --noEmit

# Lint the code
npm run lint

# Run tests (not yet implemented)
# npm test

The app uses Electron Forge with Webpack for optimal development experience:

• Hot Reload: Changes to renderer process auto-reload
• DevTools: Automatically opened in development
• Source Maps: Full debugging support
• TypeScript: Type checking and IntelliSense

digital-memory/
├── src/
│   ├── main/                 # Main process code
│   │   ├── capture.ts        # Screen capture logic
│   │   └── services/         # Main process services
│   │       └── captureCoordinator.ts # Capture coordination
│   ├── services/             # Core services
│   │   ├── database.ts       # SQLite with FTS5
│   │   ├── ocr.ts           # Tesseract.js integration
│   │   └── tavily.ts        # Tavily API client (placeholder)
│   ├── renderer/             # Renderer process code
│   │   ├── search/          # Search UI components
│   │   ├── dashboard/       # Dashboard components
│   │   └── components/      # Shared components
│   ├── types/               # TypeScript type definitions
│   │   └── index.d.ts       # Global type declarations
│   ├── utils/               # Utility functions
│   │   ├── colors.ts        # Activity color mapping
│   │   └── formatters.ts    # Time/data formatters
│   ├── index.ts             # Main process entry point
│   ├── preload.ts           # Secure IPC bridge
│   ├── renderer.ts          # Main UI logic
│   ├── index.html           # App HTML
│   └── index.css            # Styles
├── .env                     # Environment variables (optional)
├── .env.example             # Example environment config
├── forge.config.ts          # Electron Forge config
├── webpack.*.config.ts      # Webpack configurations
├── package.json             # Dependencies and scripts
└── tsconfig.json           # TypeScript config

# Create packaged app for your current platform
npm run package

# Output location
# macOS: out/digital-memory-darwin-arm64/digital-memory.app
# Windows: out/digital-memory-win32-x64/digital-memory.exe
# Linux: out/digital-memory-linux-x64/digital-memory

# Create distributable installer for current platform
npm run make

# Creates:
# macOS: .dmg installer
# Windows: .exe installer
# Linux: .deb/.rpm packages

# Build for all platforms (requires appropriate toolchain)
npm run publish

# Note: Cross-platform builds may require additional setup:
# - Windows: Wine for building on macOS/Linux
# - macOS: Requires macOS for code signing
# - Linux: Native or Docker environment

Edit forge.config.ts to customize:

• App icons and metadata
• Code signing certificates
• Auto-update configuration
• Platform-specific options

1. Download the appropriate package for your OS
2. Install following platform conventions:
   • macOS: Drag to Applications folder
   • Windows: Run installer .exe
   • Linux: Install .deb/.rpm package

npm install
npm start

1. Launch the app: It will appear in your system tray
2. Configure Tavily API:
   • Right-click tray icon → Settings
   • Enter your Tavily API key
   • Save settings
3. Start capturing:
   • Right-click tray icon → Start Capture
   • Or use the dashboard toggle

• Start/Pause Capture: Toggle screen capturing
• Open Dashboard: View timeline and statistics
• Search: Open spotlight search
• Settings: Configure API keys and preferences
• Quit: Exit application

• Cmd/Ctrl + Shift + Space: Open quick search
• Escape: Close search overlay
• Cmd/Ctrl + ,: Open settings (when focused)

The app intelligently adjusts capture frequency based on activity:

• Active (< 10s idle): 15-second intervals
• Moderate (10-60s idle): 30-second intervals
• Idle (60-300s idle): 60-second intervals
• Very Idle (> 5 min): Stops capturing automatically

• Real-time Updates: Dashboard refreshes every 30 seconds
• Secure Image Loading: Screenshots loaded via IPC for security
• OCR Processing: Automatic text extraction from screenshots
• Activity Detection: Categorizes activity (coding, browsing, etc.)

• Natural Language: "Python error from this morning"
• App Filter: "in:vscode authentication"
• Time Range: "yesterday React hooks"
• Error Detection: Automatically highlights errors

• Database: ~/Library/Application Support/digital-memory/digital-memory.db
• Screenshots: ~/Library/Application Support/digital-memory/screenshots/
• Electron Cache: ~/Library/Application Support/digital-memory/Cache/
• Settings: ~/Library/Application Support/digital-memory/Preferences

• All data stored locally by default
• Screenshots compressed with Sharp (WebP format)
• Tavily API called only when:
  • User triggers enrichment
  • URL detected in screenshot
  • Error message found
• Automatic cleanup of old data (configurable retention)

# Check for port conflicts
lsof -i :9000

# Clear cache and restart
rm -rf ~/Library/Application\ Support/digital-memory/cache

1. Grant screen recording permissions:
   • macOS: System Preferences → Security & Privacy → Screen Recording
   • Windows: Approve UAC prompt
   • Linux: May need X11 permissions

# Clear Tesseract cache
rm -rf ~/Library/Application\ Support/digital-memory/tesseract-cache

# Reinstall Tesseract data
npm rebuild tesseract.js

# Backup existing database
cp ~/Library/Application\ Support/digital-memory/digital-memory.db ./backup.db

# Reset database (loses all data)
rm ~/Library/Application\ Support/digital-memory/digital-memory.db

// In settings or .env
CAPTURE_INTERVAL=60000  // Increase to 60s for less CPU usage
OCR_ENABLED=false       // Disable OCR if not needed
IMAGE_QUALITY=60        // Reduce quality (default: 80)
MAX_IMAGE_WIDTH=1920    // Limit screenshot dimensions

-- Run via database tool
VACUUM;                 -- Optimize database
ANALYZE;               -- Update statistics
DELETE FROM screenshots WHERE timestamp < strftime('%s', 'now', '-30 days') * 1000;

1. Search API (/search)

   • Error resolution
   • Context finding
   • Daily summaries

2. Extract API (/extract)

   • URL content extraction
   • Full article retrieval
   • Documentation fetching

// Tavily integration is currently a placeholder
// The TavilyService class exists but methods are not implemented
// Future implementation will include:

// Enrich screenshot with URL content
// await tavilyService.enrichWithContext(screenshotId, urls);

// Search for error solutions
// await tavilyService.findSolution(errorText);

// Note: Enrichment can be triggered via the dashboard
// by clicking on a screenshot (calls enrichScreenshot IPC)

• Cache all Tavily responses (24-hour TTL)
• Batch enrichment requests when possible
• Use search_depth: 'basic' by default
• Manual enrichment trigger to control usage

# Enable debug logging
DEBUG=true npm start

# View Electron logs
~/Library/Logs/digital-memory/main.log

// In DevTools console
await window.electronAPI.startCapture();
// Wait a moment
const screenshots = await window.electronAPI.getRecentScreenshots(5);
console.log(screenshots);

// In development, set in tavily.ts
const MOCK_MODE = process.env.NODE_ENV === 'development';
if (MOCK_MODE) {
  return mockTavilyResponse();
}

• electron: Desktop framework
• electron-forge: Build toolchain
• typescript: Type safety

• better-sqlite3: SQLite with FTS5
• electron-store: Settings persistence

• tesseract.js: OCR processing
  • Uses eng.traineddata file (included in project root)
  • Pre-trained English language model for text recognition
  • Enables text extraction from screenshots for searchability
• screenshot-desktop: Screen capture
• Native Electron APIs for image handling

• axios: HTTP client
• dotenv: Environment config

• bootstrap: UI components
• jquery: DOM manipulation

1. Fork the repository
2. Create a feature branch
3. Make your changes
4. Add tests if applicable
5. Submit a pull request

MIT License - Created for Tavily Hackathon 2024

• Tavily API Documentation
• Electron Documentation
• Tesseract.js Guide
• Project Issues

Built with ❤️ for the Tavily Multimodal Web Agents Hackathon