qananasikq
|
133c125e9c
|
fix: eliminate greenlet crash + memory leak protection
- docker-compose: switch worker to --pool=solo --concurrency=1 --max-tasks-per-child=1
- tasks.py: remove ThreadPoolExecutor wrapper from _run_browser_job (greenlet crash)
- scraper.py: browser fallback runs sequentially instead of ThreadPoolExecutor
- scraper.py: add gc.collect() after scraper close to prevent memory leaks
|
2026-04-20 17:33:51 +03:00 |
|
qananasikq
|
3c71c098cd
|
fix: remove duplicate index=True from migration 004 columns
|
2026-04-20 17:14:33 +03:00 |
|
qananasikq
|
65d5f8e1eb
|
Refactor to new ingestion pipeline architecture with discovery, fetch, enrichment services
|
2026-04-20 16:27:42 +03:00 |
|
qananasikq
|
55203d33ea
|
chore: cleanup ad-hoc debug scripts + sitemap test downloads
|
2026-04-18 16:35:07 +03:00 |
|
qananasikq
|
5999c2e42d
|
checkpoint: pipeline + 27k/h baseline before sitemap discovery
|
2026-04-18 16:34:09 +03:00 |
|
qananasikq
|
a8c5f8aa41
|
Replace page-level checkpoint with segment-level resume
|
2026-04-17 17:51:23 +03:00 |
|
qananasikq
|
05d1ffb5ac
|
Stabilize worker sync flow
|
2026-04-17 17:30:28 +03:00 |
|
qananasikq
|
3c3aea8eb3
|
tune vps config and update tests
|
2026-04-15 21:58:10 +03:00 |
|
qananasikq
|
37d01c4ba1
|
clean mapper comments
|
2026-04-15 21:58:03 +03:00 |
|
qananasikq
|
6165c65159
|
add segmented listing sync
|
2026-04-15 21:57:40 +03:00 |
|
qananasikq
|
acbb045b6e
|
Add checkpoint logic and tests
|
2026-04-15 11:10:32 +03:00 |
|
qananasikq
|
d9111be2d2
|
fix worker lock
|
2026-04-14 19:32:45 +03:00 |
|
qananasikq
|
cac68bda4c
|
add full scan mode
|
2026-04-14 19:32:34 +03:00 |
|
qananasikq
|
0add3913eb
|
fix listing parsing
|
2026-04-14 19:31:57 +03:00 |
|
qananasikq
|
6a334d3c64
|
tune docker config
|
2026-04-14 14:03:26 +03:00 |
|
qananasikq
|
d51216ddb7
|
improve listing sync
|
2026-04-14 14:03:16 +03:00 |
|
qananasikq
|
9337344182
|
stabilize worker
|
2026-04-14 14:03:08 +03:00 |
|
qananasikq
|
c729f971e3
|
fix worker recovery and docker logs
|
2026-04-13 20:49:25 +03:00 |
|
|
|
eaafbd5816
|
Удалить .env
|
2026-04-13 19:46:28 +02:00 |
|
qananasikq
|
a4c93a1df1
|
fix docker scraping
improve batch sync
add postgres upsert
fix sync locking
improve listing sync
speed up scraper
clean up project
prepare for github
update docker setup
|
2026-04-13 16:35:08 +03:00 |
|
qananasikq
|
b9557922ed
|
fix parser_id format: car- + 22 alphanum chars, origin_id: iaai:{lot}
|
2026-04-10 20:31:48 +03:00 |
|
qananasikq
|
85b4ff8502
|
remove xvfb, use headless with cookie auth
|
2026-04-10 20:01:53 +03:00 |
|
qananasikq
|
6134b10fd0
|
add iaai auto-login with session persistence
|
2026-04-10 19:59:03 +03:00 |
|
qananasikq
|
05b83b5518
|
add cookie session persistence
|
2026-04-10 19:55:38 +03:00 |
|
qananasikq
|
b1aeb966ac
|
add sqlite check command to readme
|
2026-04-10 19:51:38 +03:00 |
|
qananasikq
|
1acfafe665
|
remove github workflow
|
2026-04-10 19:49:24 +03:00 |
|
qananasikq
|
2cbefbde93
|
update local run setup
ci / test (push) Has been cancelled
|
2026-04-10 19:47:20 +03:00 |
|
qananasikq
|
db2fa5ec17
|
add antibot unit tests
|
2026-04-10 15:18:57 +03:00 |
|
qananasikq
|
18dab6d48d
|
docker non-root and healthcheck fix
|
2026-04-10 15:18:57 +03:00 |
|
qananasikq
|
3870cf4d94
|
add task status endpoint
|
2026-04-10 15:18:57 +03:00 |
|
qananasikq
|
62aafae793
|
add Redis lock for sync_listing
|
2026-04-10 15:18:57 +03:00 |
|
qananasikq
|
acf30fcc20
|
add antibot guard and retry
|
2026-04-10 15:18:57 +03:00 |
|
qananasikq
|
7b1501008e
|
fix limit after only_new filter
|
2026-04-10 15:18:57 +03:00 |
|
|
|
6d1702faae
|
Обновить README.md
|
2026-04-10 10:08:33 +02:00 |
|
qananasikq
|
2931eee0a7
|
fix readme
|
2026-04-09 20:35:41 +03:00 |
|
qananasikq
|
f6d7c8ea60
|
cleanup and fix runtime bugs
|
2026-04-09 20:35:25 +03:00 |
|
qananasikq
|
eb9fb48ea0
|
improve docker compose setup
|
2026-04-09 20:34:34 +03:00 |
|
qananasikq
|
042ffdcfbb
|
move deps to pyproject
|
2026-04-09 20:34:20 +03:00 |
|
qananasikq
|
437aedad45
|
update celery schedule and readme
|
2026-04-08 23:46:13 +03:00 |
|
qananasikq
|
9f703696d8
|
fix readme
|
2026-04-08 23:43:21 +03:00 |
|
qananasikq
|
3992067a94
|
rewrite readme
|
2026-04-08 22:55:00 +03:00 |
|
qananasikq
|
e5700f9623
|
postgres redis docker compose setup
|
2026-04-08 22:54:51 +03:00 |
|
qananasikq
|
f4b9d20191
|
add fastapi rest api and celery workers
|
2026-04-08 22:54:33 +03:00 |
|
qananasikq
|
6e97991c68
|
cleanup and fix runtime bugs
|
2026-04-08 22:54:16 +03:00 |
|
qananasikq
|
90bd4756b3
|
update readme and env
|
2026-04-08 18:33:56 +03:00 |
|
qananasikq
|
64b7c760d9
|
update tests for sync
|
2026-04-08 18:32:45 +03:00 |
|
qananasikq
|
f161071e07
|
harden price normalization
|
2026-04-08 18:30:39 +03:00 |
|
qananasikq
|
3b218534ec
|
sync only new cars
|
2026-04-08 18:30:02 +03:00 |
|
qananasikq
|
54fea100dc
|
add docker proxy bridge
|
2026-04-08 18:29:06 +03:00 |
|
qananasikq
|
447a88feed
|
persist raw attributes and tighten sync assertions
|
2026-04-08 14:30:57 +03:00 |
|
