feat(console): add glossary:bulk-resync, ~250x faster than glossary:resync (#1428)
Commit: ec388007a91af840df50193df5fdf80324c7d645
Date: 2026-07-06 06:18:41 +0000
Author: PascalHavelange
Commit Message
feat(console): add glossary:bulk-resync, ~250x faster than glossary:resync (#1428)
glossary:resync dispatches one job per glossary spelling (thousands), and
each job re-scans every translation of that language from scratch --
O(spellings x translations). Against the real dataset this takes over a
day and was still only ~27% done after 9 hours in production use.
glossary:bulk-resync inverts the loop: one combined, escaped, longest-match-
first regex per language, one pass per translation, syncing all matched
spelling ids in a single write. Verified against the full local-mysql
dataset (67,301 translations across 10 languages): 87s-466s depending on
cache state, versus an estimated 30+ hours for glossary:resync to finish
the same work -- and produces identical links for spot-checked terms
(e.g. "Hegira" links the same 6 languages both ways).
Runs synchronously, no queue involved. local-glossary-sync now calls it
instead of glossary:resync + queue:work.
Also documents (via a dedicated test, not a fix) a pre-existing limitation
shared with the legacy per-spelling job: both wrap every spelling in
\b...\b, and \b can never match immediately after a non-word character, so
any spelling ending in punctuation (e.g. the real "Gilding (gold)" glossary
entry) can never match anywhere regardless of which sync implementation
runs. Not a regression introduced here.
EOF
Co-authored-by: Pascal HAVELANGE <havelangep@hotmail.com>
Files Changed
- ✅ Added:
app/Console/Commands/GlossaryBulkResync.php - 📝 Modified:
scripts/import-tool/README.md - 📝 Modified:
scripts/import-tool/docker-compose.yml - ✅ Added:
tests/Console/GlossaryBulkResyncTest.php
Links
This documentation was automatically generated from Git commit data.