Commit Graph

38 Commits

Author SHA1 Message Date
qananasikq
a4c93a1df1 fix docker scraping
improve batch sync

add postgres upsert

fix sync locking

improve listing sync

speed up scraper

clean up project

prepare for github

update docker setup
2026-04-13 16:35:08 +03:00
qananasikq
b9557922ed fix parser_id format: car- + 22 alphanum chars, origin_id: iaai:{lot} 2026-04-10 20:31:48 +03:00
qananasikq
85b4ff8502 remove xvfb, use headless with cookie auth 2026-04-10 20:01:53 +03:00
qananasikq
6134b10fd0 add iaai auto-login with session persistence 2026-04-10 19:59:03 +03:00
qananasikq
05b83b5518 add cookie session persistence 2026-04-10 19:55:38 +03:00
qananasikq
b1aeb966ac add sqlite check command to readme 2026-04-10 19:51:38 +03:00
qananasikq
1acfafe665 remove github workflow 2026-04-10 19:49:24 +03:00
qananasikq
2cbefbde93 update local run setup 2026-04-10 19:47:20 +03:00
qananasikq
db2fa5ec17 add antibot unit tests 2026-04-10 15:18:57 +03:00
qananasikq
18dab6d48d docker non-root and healthcheck fix 2026-04-10 15:18:57 +03:00
qananasikq
3870cf4d94 add task status endpoint 2026-04-10 15:18:57 +03:00
qananasikq
62aafae793 add Redis lock for sync_listing 2026-04-10 15:18:57 +03:00
qananasikq
acf30fcc20 add antibot guard and retry 2026-04-10 15:18:57 +03:00
qananasikq
7b1501008e fix limit after only_new filter 2026-04-10 15:18:57 +03:00
6d1702faae Обновить README.md 2026-04-10 10:08:33 +02:00
qananasikq
2931eee0a7 fix readme 2026-04-09 20:35:41 +03:00
qananasikq
f6d7c8ea60 cleanup and fix runtime bugs 2026-04-09 20:35:25 +03:00
qananasikq
eb9fb48ea0 improve docker compose setup 2026-04-09 20:34:34 +03:00
qananasikq
042ffdcfbb move deps to pyproject 2026-04-09 20:34:20 +03:00
qananasikq
437aedad45 update celery schedule and readme 2026-04-08 23:46:13 +03:00
qananasikq
9f703696d8 fix readme 2026-04-08 23:43:21 +03:00
qananasikq
3992067a94 rewrite readme 2026-04-08 22:55:00 +03:00
qananasikq
e5700f9623 postgres redis docker compose setup 2026-04-08 22:54:51 +03:00
qananasikq
f4b9d20191 add fastapi rest api and celery workers 2026-04-08 22:54:33 +03:00
qananasikq
6e97991c68 cleanup and fix runtime bugs 2026-04-08 22:54:16 +03:00
qananasikq
90bd4756b3 update readme and env 2026-04-08 18:33:56 +03:00
qananasikq
64b7c760d9 update tests for sync 2026-04-08 18:32:45 +03:00
qananasikq
f161071e07 harden price normalization 2026-04-08 18:30:39 +03:00
qananasikq
3b218534ec sync only new cars 2026-04-08 18:30:02 +03:00
qananasikq
54fea100dc add docker proxy bridge 2026-04-08 18:29:06 +03:00
qananasikq
447a88feed persist raw attributes and tighten sync assertions 2026-04-08 14:30:57 +03:00
qananasikq
b2d3902bcc stabilize daemon sync and docker startup 2026-04-08 14:30:44 +03:00
qananasikq
d1844ba625 document docker runtime and tune env defaults 2026-04-08 14:30:27 +03:00
ff2a1c7ac8 Обновить README.md 2026-04-08 12:38:15 +02:00
qananasikq
0010abdf08 fix parsing db and tests 2026-04-08 13:31:02 +03:00
qananasikq
f96e5ca5f8 improve scraper runtime 2026-04-08 13:30:45 +03:00
qananasikq
21bdf17f35 update readme and env 2026-04-08 13:30:17 +03:00
qananasikq
78b52b265f IAAI scraper 2026-04-07 23:51:41 +03:00