qananasikq
|
9f215dc70a
|
Update parser
|
2026-04-27 20:47:44 +03:00 |
|
qananasikq
|
d937aa2386
|
Add worker fast sync module
|
2026-04-27 13:09:45 +03:00 |
|
qananasikq
|
1168ed8d4f
|
Harden fast sync progress logging
|
2026-04-27 13:09:41 +03:00 |
|
qananasikq
|
993f692254
|
Fix self heal stale progress cleanup
|
2026-04-27 13:09:34 +03:00 |
|
qananasikq
|
331d8d350e
|
fix: beat schedule should use only_new=False for full scans
|
2026-04-25 14:55:36 +03:00 |
|
qananasikq
|
f8206615a9
|
add vps check
|
2026-04-24 18:36:58 +03:00 |
|
qananasikq
|
31f87a9bdf
|
update parser
|
2026-04-24 18:36:31 +03:00 |
|
qananasikq
|
1453eee83b
|
update sync
|
2026-04-24 18:00:09 +03:00 |
|
|
|
8c441b2aec
|
Fixed proxies
|
2026-04-24 13:43:30 +03:00 |
|
qananasikq
|
fc1bea562a
|
fast parser
|
2026-04-24 12:39:05 +03:00 |
|
|
|
3a3222c7dc
|
Fixed queue name
|
2026-04-23 23:30:07 +03:00 |
|
|
|
11b0db5560
|
Merge pull request 'feat: Prepared for production' (#1) from prod-preparation into main
Reviewed-on: #1
|
2026-04-23 22:25:06 +02:00 |
|
|
|
6b69b8c002
|
Merge branch 'main' into prod-preparation
|
2026-04-23 22:24:53 +02:00 |
|
qananasikq
|
e726461e75
|
fix self heal restart
|
2026-04-23 21:34:57 +03:00 |
|
|
|
f221aebef0
|
Prepared for production
|
2026-04-23 13:54:53 +03:00 |
|
qananasikq
|
6aba4f0dbd
|
fix resume
|
2026-04-23 09:56:18 +03:00 |
|
qananasikq
|
09fecdae31
|
fix watchdog
|
2026-04-23 09:56:18 +03:00 |
|
qananasikq
|
d5d6ba8b7c
|
fix scraper flow
|
2026-04-22 21:40:40 +03:00 |
|
qananasikq
|
133c125e9c
|
fix: eliminate greenlet crash + memory leak protection
- docker-compose: switch worker to --pool=solo --concurrency=1 --max-tasks-per-child=1
- tasks.py: remove ThreadPoolExecutor wrapper from _run_browser_job (greenlet crash)
- scraper.py: browser fallback runs sequentially instead of ThreadPoolExecutor
- scraper.py: add gc.collect() after scraper close to prevent memory leaks
|
2026-04-20 17:33:51 +03:00 |
|
qananasikq
|
3c71c098cd
|
fix: remove duplicate index=True from migration 004 columns
|
2026-04-20 17:14:33 +03:00 |
|
qananasikq
|
65d5f8e1eb
|
Refactor to new ingestion pipeline architecture with discovery, fetch, enrichment services
|
2026-04-20 16:27:42 +03:00 |
|
qananasikq
|
55203d33ea
|
chore: cleanup ad-hoc debug scripts + sitemap test downloads
|
2026-04-18 16:35:07 +03:00 |
|
qananasikq
|
5999c2e42d
|
checkpoint: pipeline + 27k/h baseline before sitemap discovery
|
2026-04-18 16:34:09 +03:00 |
|
qananasikq
|
a8c5f8aa41
|
Replace page-level checkpoint with segment-level resume
|
2026-04-17 17:51:23 +03:00 |
|
qananasikq
|
05d1ffb5ac
|
Stabilize worker sync flow
|
2026-04-17 17:30:28 +03:00 |
|
qananasikq
|
3c3aea8eb3
|
tune vps config and update tests
|
2026-04-15 21:58:10 +03:00 |
|
qananasikq
|
37d01c4ba1
|
clean mapper comments
|
2026-04-15 21:58:03 +03:00 |
|
qananasikq
|
6165c65159
|
add segmented listing sync
|
2026-04-15 21:57:40 +03:00 |
|
qananasikq
|
acbb045b6e
|
Add checkpoint logic and tests
|
2026-04-15 11:10:32 +03:00 |
|
qananasikq
|
d9111be2d2
|
fix worker lock
|
2026-04-14 19:32:45 +03:00 |
|
qananasikq
|
cac68bda4c
|
add full scan mode
|
2026-04-14 19:32:34 +03:00 |
|
qananasikq
|
0add3913eb
|
fix listing parsing
|
2026-04-14 19:31:57 +03:00 |
|
qananasikq
|
6a334d3c64
|
tune docker config
|
2026-04-14 14:03:26 +03:00 |
|
qananasikq
|
d51216ddb7
|
improve listing sync
|
2026-04-14 14:03:16 +03:00 |
|
qananasikq
|
9337344182
|
stabilize worker
|
2026-04-14 14:03:08 +03:00 |
|
qananasikq
|
c729f971e3
|
fix worker recovery and docker logs
|
2026-04-13 20:49:25 +03:00 |
|
|
|
eaafbd5816
|
Удалить .env
|
2026-04-13 19:46:28 +02:00 |
|
qananasikq
|
a4c93a1df1
|
fix docker scraping
improve batch sync
add postgres upsert
fix sync locking
improve listing sync
speed up scraper
clean up project
prepare for github
update docker setup
|
2026-04-13 16:35:08 +03:00 |
|
qananasikq
|
b9557922ed
|
fix parser_id format: car- + 22 alphanum chars, origin_id: iaai:{lot}
|
2026-04-10 20:31:48 +03:00 |
|
qananasikq
|
85b4ff8502
|
remove xvfb, use headless with cookie auth
|
2026-04-10 20:01:53 +03:00 |
|
qananasikq
|
6134b10fd0
|
add iaai auto-login with session persistence
|
2026-04-10 19:59:03 +03:00 |
|
qananasikq
|
05b83b5518
|
add cookie session persistence
|
2026-04-10 19:55:38 +03:00 |
|
qananasikq
|
b1aeb966ac
|
add sqlite check command to readme
|
2026-04-10 19:51:38 +03:00 |
|
qananasikq
|
1acfafe665
|
remove github workflow
|
2026-04-10 19:49:24 +03:00 |
|
qananasikq
|
2cbefbde93
|
update local run setup
ci / test (push) Has been cancelled
|
2026-04-10 19:47:20 +03:00 |
|
qananasikq
|
db2fa5ec17
|
add antibot unit tests
|
2026-04-10 15:18:57 +03:00 |
|
qananasikq
|
18dab6d48d
|
docker non-root and healthcheck fix
|
2026-04-10 15:18:57 +03:00 |
|
qananasikq
|
3870cf4d94
|
add task status endpoint
|
2026-04-10 15:18:57 +03:00 |
|
qananasikq
|
62aafae793
|
add Redis lock for sync_listing
|
2026-04-10 15:18:57 +03:00 |
|
qananasikq
|
acf30fcc20
|
add antibot guard and retry
|
2026-04-10 15:18:57 +03:00 |
|
