Commit Graph

13 Commits

Author SHA1 Message Date
qananasikq
7b7efbf525 harden: cache PG/Redis singletons, safe orphan-lock cleanup, ScraperAbortedError on lock loss 2026-04-18 00:33:10 +03:00
qananasikq
a8c5f8aa41 Replace page-level checkpoint with segment-level resume 2026-04-17 17:51:23 +03:00
qananasikq
05d1ffb5ac Stabilize worker sync flow 2026-04-17 17:30:28 +03:00
qananasikq
6165c65159 add segmented listing sync 2026-04-15 21:57:40 +03:00
qananasikq
acbb045b6e Add checkpoint logic and tests 2026-04-15 11:10:32 +03:00
qananasikq
d9111be2d2 fix worker lock 2026-04-14 19:32:45 +03:00
qananasikq
9337344182 stabilize worker 2026-04-14 14:03:08 +03:00
qananasikq
c729f971e3 fix worker recovery and docker logs 2026-04-13 20:49:25 +03:00
qananasikq
a4c93a1df1 fix docker scraping
improve batch sync

add postgres upsert

fix sync locking

improve listing sync

speed up scraper

clean up project

prepare for github

update docker setup
2026-04-13 16:35:08 +03:00
qananasikq
62aafae793 add Redis lock for sync_listing 2026-04-10 15:18:57 +03:00
qananasikq
f6d7c8ea60 cleanup and fix runtime bugs 2026-04-09 20:35:25 +03:00
qananasikq
437aedad45 update celery schedule and readme 2026-04-08 23:46:13 +03:00
qananasikq
f4b9d20191 add fastapi rest api and celery workers 2026-04-08 22:54:33 +03:00