Skip to content

Latest commit

 

History

History
executable file
·
170 lines (116 loc) · 3.84 KB

2021-12-trace.moe.md

File metadata and controls

executable file
·
170 lines (116 loc) · 3.84 KB
marp size html header footer
true
14580
true
<style> footer { color: #999; font-size: 12px; } </style>

Anime Scene Search Engine (2021)

trace.moe 2021 Project Updates

All presentations slides: https://github.com/soruly/slides

Boring Stats

  • 12 million total page views (GA) +7.5% YoY
  • ? million search served by Telegram Bot
  • ? million API search served for 3rd party apps/bots
  • 11k Web Extension users (10k Chrome, Firefox+Opera+Edge: <2k)
  • 130k monthly active users, 6k daily users (GA)
  • 450k monthly unique visiters, 20k daily (Cloudflare)
  • 22k daily search

Updates Summary of 2021

  • New website design
  • MS Edge Extension
  • trace.moe API standardized and out of beta
  • Rate limit and queue system
  • Search quota
  • User accounts for donator
  • New backend infrastructure

Website redesign

  • Nextjs
  • SFW by default, video preview of all results, concise anime info
  • Cloudflare Pages + Cloudflare Workers (for image proxy and title translation)
  • Search traffic and search time
  • Database statistics
  • New status page

https://github.com/soruly/trace.moe-www


MS Edge Extension

https://microsoftedge.microsoft.com/addons/detail/search-anime-by-screensho/bkigcpancdclbiekidfbcghedaielbda


Windows Explorer Context Menu

https://github.com/soruly/trace.moe-windows-menu


Telegram Bot

Linode => Vercel => Heroku / Railway

https://github.com/soruly/trace.moe-telegram-bot-vercel


data size

  • 5162 => 5604 anime series
  • 29256 => 31259 hours, 21TB = >23TB
  • 2.7 billion frames analyzed, 772 million indexed (156GB in size)

Analyzing all anime takes 2x3950X+1x3700X 75 hours to complete (peak ~10000 hours/day)

https://trace.moe/about


API

https://github.com/soruly/trace.moe-api


New Backend Infrastructure

  • Tidy up a lot 🥰
  • fully containerized
  • Updated LIRE, Solr, Java (8=>17)
  • RabbitMQ => WebSocket
  • CI tests

Previously Planned Works

  • Migrate from PHP to nodejs ✅
  • Search queue w/ priority ✅
  • Test cases ✅
  • Migrate to cloud partially 🥴
  • New server ⌛

New Planned Works

  • Prepare for Web Extension Manifest V3
  • Index Youtube channels automatically
  • Detect and store scene boundaries up-front (with Rust?)
  • Better search image manipulation (like black border crop)
  • Cross reference results with JCD if new server is available
  • Match video to video

Project URLs


Official Channels


Credits

  • Dr. Mathias Lux for LIRE Project and liresolr
  • anilist.co for providing anime info via Anilist API
  • Sponsors from Patreon, GitHub and Paypal
  • fans who helped answering questions in various channels
  • friends who reported bugs and issues to me

https://about.me/soruly