qananasikq
|
133c125e9c
|
fix: eliminate greenlet crash + memory leak protection
- docker-compose: switch worker to --pool=solo --concurrency=1 --max-tasks-per-child=1
- tasks.py: remove ThreadPoolExecutor wrapper from _run_browser_job (greenlet crash)
- scraper.py: browser fallback runs sequentially instead of ThreadPoolExecutor
- scraper.py: add gc.collect() after scraper close to prevent memory leaks
|
2026-04-20 17:33:51 +03:00 |
|
qananasikq
|
65d5f8e1eb
|
Refactor to new ingestion pipeline architecture with discovery, fetch, enrichment services
|
2026-04-20 16:27:42 +03:00 |
|
qananasikq
|
5999c2e42d
|
checkpoint: pipeline + 27k/h baseline before sitemap discovery
|
2026-04-18 16:34:09 +03:00 |
|
qananasikq
|
a8c5f8aa41
|
Replace page-level checkpoint with segment-level resume
|
2026-04-17 17:51:23 +03:00 |
|
qananasikq
|
05d1ffb5ac
|
Stabilize worker sync flow
|
2026-04-17 17:30:28 +03:00 |
|
qananasikq
|
6165c65159
|
add segmented listing sync
|
2026-04-15 21:57:40 +03:00 |
|
qananasikq
|
acbb045b6e
|
Add checkpoint logic and tests
|
2026-04-15 11:10:32 +03:00 |
|
qananasikq
|
cac68bda4c
|
add full scan mode
|
2026-04-14 19:32:34 +03:00 |
|
qananasikq
|
d51216ddb7
|
improve listing sync
|
2026-04-14 14:03:16 +03:00 |
|
qananasikq
|
a4c93a1df1
|
fix docker scraping
improve batch sync
add postgres upsert
fix sync locking
improve listing sync
speed up scraper
clean up project
prepare for github
update docker setup
|
2026-04-13 16:35:08 +03:00 |
|
qananasikq
|
6134b10fd0
|
add iaai auto-login with session persistence
|
2026-04-10 19:59:03 +03:00 |
|
qananasikq
|
05b83b5518
|
add cookie session persistence
|
2026-04-10 19:55:38 +03:00 |
|
qananasikq
|
7b1501008e
|
fix limit after only_new filter
|
2026-04-10 15:18:57 +03:00 |
|
qananasikq
|
f6d7c8ea60
|
cleanup and fix runtime bugs
|
2026-04-09 20:35:25 +03:00 |
|
qananasikq
|
6e97991c68
|
cleanup and fix runtime bugs
|
2026-04-08 22:54:16 +03:00 |
|
qananasikq
|
3b218534ec
|
sync only new cars
|
2026-04-08 18:30:02 +03:00 |
|
qananasikq
|
b2d3902bcc
|
stabilize daemon sync and docker startup
|
2026-04-08 14:30:44 +03:00 |
|
qananasikq
|
f96e5ca5f8
|
improve scraper runtime
|
2026-04-08 13:30:45 +03:00 |
|
qananasikq
|
78b52b265f
|
IAAI scraper
|
2026-04-07 23:51:41 +03:00 |
|