Commit Graph

40 Commits

Author SHA1 Message Date
qananasikq
4096d29a07 add encar scraper 2026-04-16 18:01:05 +03:00
qananasikq
5624c1973a remove iaai scraper and old tests 2026-04-16 18:00:54 +03:00
qananasikq
a4c93a1df1 fix docker scraping
improve batch sync

add postgres upsert

fix sync locking

improve listing sync

speed up scraper

clean up project

prepare for github

update docker setup
2026-04-13 16:35:08 +03:00
qananasikq
b9557922ed fix parser_id format: car- + 22 alphanum chars, origin_id: iaai:{lot} 2026-04-10 20:31:48 +03:00
qananasikq
85b4ff8502 remove xvfb, use headless with cookie auth 2026-04-10 20:01:53 +03:00
qananasikq
6134b10fd0 add iaai auto-login with session persistence 2026-04-10 19:59:03 +03:00
qananasikq
05b83b5518 add cookie session persistence 2026-04-10 19:55:38 +03:00
qananasikq
b1aeb966ac add sqlite check command to readme 2026-04-10 19:51:38 +03:00
qananasikq
1acfafe665 remove github workflow 2026-04-10 19:49:24 +03:00
qananasikq
2cbefbde93 update local run setup 2026-04-10 19:47:20 +03:00
qananasikq
db2fa5ec17 add antibot unit tests 2026-04-10 15:18:57 +03:00
qananasikq
18dab6d48d docker non-root and healthcheck fix 2026-04-10 15:18:57 +03:00
qananasikq
3870cf4d94 add task status endpoint 2026-04-10 15:18:57 +03:00
qananasikq
62aafae793 add Redis lock for sync_listing 2026-04-10 15:18:57 +03:00
qananasikq
acf30fcc20 add antibot guard and retry 2026-04-10 15:18:57 +03:00
qananasikq
7b1501008e fix limit after only_new filter 2026-04-10 15:18:57 +03:00
6d1702faae Обновить README.md 2026-04-10 10:08:33 +02:00
qananasikq
2931eee0a7 fix readme 2026-04-09 20:35:41 +03:00
qananasikq
f6d7c8ea60 cleanup and fix runtime bugs 2026-04-09 20:35:25 +03:00
qananasikq
eb9fb48ea0 improve docker compose setup 2026-04-09 20:34:34 +03:00
qananasikq
042ffdcfbb move deps to pyproject 2026-04-09 20:34:20 +03:00
qananasikq
437aedad45 update celery schedule and readme 2026-04-08 23:46:13 +03:00
qananasikq
9f703696d8 fix readme 2026-04-08 23:43:21 +03:00
qananasikq
3992067a94 rewrite readme 2026-04-08 22:55:00 +03:00
qananasikq
e5700f9623 postgres redis docker compose setup 2026-04-08 22:54:51 +03:00
qananasikq
f4b9d20191 add fastapi rest api and celery workers 2026-04-08 22:54:33 +03:00
qananasikq
6e97991c68 cleanup and fix runtime bugs 2026-04-08 22:54:16 +03:00
qananasikq
90bd4756b3 update readme and env 2026-04-08 18:33:56 +03:00
qananasikq
64b7c760d9 update tests for sync 2026-04-08 18:32:45 +03:00
qananasikq
f161071e07 harden price normalization 2026-04-08 18:30:39 +03:00
qananasikq
3b218534ec sync only new cars 2026-04-08 18:30:02 +03:00
qananasikq
54fea100dc add docker proxy bridge 2026-04-08 18:29:06 +03:00
qananasikq
447a88feed persist raw attributes and tighten sync assertions 2026-04-08 14:30:57 +03:00
qananasikq
b2d3902bcc stabilize daemon sync and docker startup 2026-04-08 14:30:44 +03:00
qananasikq
d1844ba625 document docker runtime and tune env defaults 2026-04-08 14:30:27 +03:00
ff2a1c7ac8 Обновить README.md 2026-04-08 12:38:15 +02:00
qananasikq
0010abdf08 fix parsing db and tests 2026-04-08 13:31:02 +03:00
qananasikq
f96e5ca5f8 improve scraper runtime 2026-04-08 13:30:45 +03:00
qananasikq
21bdf17f35 update readme and env 2026-04-08 13:30:17 +03:00
qananasikq
78b52b265f IAAI scraper 2026-04-07 23:51:41 +03:00