qananasikq/iaai-parser
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity

Commit Graph

  • d9c95b1a9f Add tools main qananasikq 2026-04-27 20:47:44 +03:00
  • 00d33d527c Update parser qananasikq 2026-04-27 20:47:44 +03:00
  • f278bd1a47 Add worker fast sync module qananasikq 2026-04-27 13:09:45 +03:00
  • 0905472991 Harden fast sync progress logging qananasikq 2026-04-27 13:09:41 +03:00
  • 9a33efa89b Fix self heal stale progress cleanup qananasikq 2026-04-27 13:09:34 +03:00
  • f8206615a9 add vps check qananasikq 2026-04-24 18:36:58 +03:00
  • 31f87a9bdf update parser qananasikq 2026-04-24 18:36:31 +03:00
  • 1453eee83b update sync qananasikq 2026-04-24 18:00:09 +03:00
  • 8c441b2aec Fixed proxies duuuuuuuden 2026-04-24 13:43:30 +03:00
  • fc1bea562a fast parser qananasikq 2026-04-24 12:39:05 +03:00
  • 3a3222c7dc Fixed queue name duuuuuuuden 2026-04-23 23:30:07 +03:00
  • 11b0db5560 Merge pull request 'feat: Prepared for production' (#1) from prod-preparation into main Andrey Shershnev 2026-04-23 22:25:06 +02:00
  • 6b69b8c002 Merge branch 'main' into prod-preparation prod-preparation Andrey Shershnev 2026-04-23 22:24:53 +02:00
  • e726461e75 fix self heal restart qananasikq 2026-04-23 21:34:57 +03:00
  • f221aebef0 Prepared for production duuuuuuuden 2026-04-23 13:54:53 +03:00
  • 6aba4f0dbd fix resume qananasikq 2026-04-23 09:56:18 +03:00
  • 09fecdae31 fix watchdog qananasikq 2026-04-23 09:56:18 +03:00
  • d5d6ba8b7c fix scraper flow qananasikq 2026-04-22 21:40:40 +03:00
  • 133c125e9c fix: eliminate greenlet crash + memory leak protection qananasikq 2026-04-20 17:33:51 +03:00
  • 3c71c098cd fix: remove duplicate index=True from migration 004 columns backup-before-rollback-main-20260421 qananasikq 2026-04-20 17:14:33 +03:00
  • 65d5f8e1eb Refactor to new ingestion pipeline architecture with discovery, fetch, enrichment services backup-before-remove-yesterday-20260421 qananasikq 2026-04-20 16:27:42 +03:00
  • 55203d33ea chore: cleanup ad-hoc debug scripts + sitemap test downloads qananasikq 2026-04-18 16:35:07 +03:00
  • 5999c2e42d checkpoint: pipeline + 27k/h baseline before sitemap discovery qananasikq 2026-04-18 16:34:09 +03:00
  • a8c5f8aa41 Replace page-level checkpoint with segment-level resume qananasikq 2026-04-17 17:51:23 +03:00
  • 05d1ffb5ac Stabilize worker sync flow qananasikq 2026-04-17 17:30:28 +03:00
  • 3c3aea8eb3 tune vps config and update tests qananasikq 2026-04-15 21:58:10 +03:00
  • 37d01c4ba1 clean mapper comments qananasikq 2026-04-15 21:58:03 +03:00
  • 6165c65159 add segmented listing sync qananasikq 2026-04-15 21:57:40 +03:00
  • acbb045b6e Add checkpoint logic and tests qananasikq 2026-04-15 11:10:32 +03:00
  • d9111be2d2 fix worker lock qananasikq 2026-04-14 19:32:45 +03:00
  • cac68bda4c add full scan mode qananasikq 2026-04-14 19:32:34 +03:00
  • 0add3913eb fix listing parsing qananasikq 2026-04-14 19:31:57 +03:00
  • 6a334d3c64 tune docker config qananasikq 2026-04-14 14:03:26 +03:00
  • d51216ddb7 improve listing sync qananasikq 2026-04-14 14:03:16 +03:00
  • 9337344182 stabilize worker qananasikq 2026-04-14 14:03:08 +03:00
  • c729f971e3 fix worker recovery and docker logs qananasikq 2026-04-13 20:46:39 +03:00
  • eaafbd5816 Удалить .env qananasikq 2026-04-13 19:46:28 +02:00
  • a4c93a1df1 fix docker scraping qananasikq 2026-04-13 16:35:08 +03:00
  • b9557922ed fix parser_id format: car- + 22 alphanum chars, origin_id: iaai:{lot} qananasikq 2026-04-10 20:31:48 +03:00
  • 85b4ff8502 remove xvfb, use headless with cookie auth qananasikq 2026-04-10 20:01:53 +03:00
  • 6134b10fd0 add iaai auto-login with session persistence qananasikq 2026-04-10 19:59:03 +03:00
  • 05b83b5518 add cookie session persistence qananasikq 2026-04-10 19:55:38 +03:00
  • b1aeb966ac add sqlite check command to readme qananasikq 2026-04-10 19:51:38 +03:00
  • 1acfafe665 remove github workflow qananasikq 2026-04-10 19:49:24 +03:00
  • 2cbefbde93 update local run setup qananasikq 2026-04-10 19:46:02 +03:00
  • db2fa5ec17 add antibot unit tests qananasikq 2026-04-10 15:15:07 +03:00
  • 18dab6d48d docker non-root and healthcheck fix qananasikq 2026-04-10 15:14:04 +03:00
  • 3870cf4d94 add task status endpoint qananasikq 2026-04-10 15:13:33 +03:00
  • 62aafae793 add Redis lock for sync_listing qananasikq 2026-04-10 15:12:42 +03:00
  • acf30fcc20 add antibot guard and retry qananasikq 2026-04-10 15:10:50 +03:00
  • 7b1501008e fix limit after only_new filter qananasikq 2026-04-10 15:10:38 +03:00
  • 6d1702faae Обновить README.md qananasikq 2026-04-10 10:08:33 +02:00
  • 2931eee0a7 fix readme qananasikq 2026-04-09 20:35:41 +03:00
  • f6d7c8ea60 cleanup and fix runtime bugs qananasikq 2026-04-09 20:34:48 +03:00
  • eb9fb48ea0 improve docker compose setup qananasikq 2026-04-09 20:34:34 +03:00
  • 042ffdcfbb move deps to pyproject qananasikq 2026-04-09 20:34:20 +03:00
  • 437aedad45 update celery schedule and readme qananasikq 2026-04-08 23:46:13 +03:00
  • 9f703696d8 fix readme qananasikq 2026-04-08 23:43:21 +03:00
  • 3992067a94 rewrite readme qananasikq 2026-04-08 22:55:00 +03:00
  • e5700f9623 postgres redis docker compose setup qananasikq 2026-04-08 22:54:51 +03:00
  • f4b9d20191 add fastapi rest api and celery workers qananasikq 2026-04-08 22:54:33 +03:00
  • 6e97991c68 cleanup and fix runtime bugs qananasikq 2026-04-08 22:54:16 +03:00
  • 90bd4756b3 update readme and env qananasikq 2026-04-08 18:33:56 +03:00
  • 64b7c760d9 update tests for sync qananasikq 2026-04-08 18:32:45 +03:00
  • f161071e07 harden price normalization qananasikq 2026-04-08 18:30:39 +03:00
  • 3b218534ec sync only new cars qananasikq 2026-04-08 18:30:02 +03:00
  • 54fea100dc add docker proxy bridge proxy-bridge qananasikq 2026-04-08 18:29:06 +03:00
  • 447a88feed persist raw attributes and tighten sync assertions qananasikq 2026-04-08 14:30:57 +03:00
  • b2d3902bcc stabilize daemon sync and docker startup qananasikq 2026-04-08 14:30:44 +03:00
  • d1844ba625 document docker runtime and tune env defaults qananasikq 2026-04-08 14:30:27 +03:00
  • ff2a1c7ac8 Обновить README.md qananasikq 2026-04-08 12:38:15 +02:00
  • 0010abdf08 fix parsing db and tests qananasikq 2026-04-08 13:31:02 +03:00
  • f96e5ca5f8 improve scraper runtime qananasikq 2026-04-08 13:30:45 +03:00
  • 21bdf17f35 update readme and env qananasikq 2026-04-08 13:30:17 +03:00
  • 78b52b265f IAAI scraper qananasikq 2026-04-07 23:51:41 +03:00