Skip to content

perf(queries): index migration + tz-aware days (closes #208) - #209

Draft
carvalab wants to merge 1 commit into
FreeOpenSourcePOS:mainfrom
carvalab:find-issue-performance
Draft

perf(queries): index migration + tz-aware days (closes #208)#209
carvalab wants to merge 1 commit into
FreeOpenSourcePOS:mainfrom
carvalab:find-issue-performance

Conversation

@carvalab

@carvalab carvalab commented Aug 1, 2026

Copy link
Copy Markdown
Contributor

Closes #208.

I built a synthetic 1-year database (109,500 orders at 300/day, plus a 3-year at 328,500) and timed every hot query on this branch against the same dataset the issue measured. Everything below was measured, not estimated.

What this PR does:

  • New migration v40 adds 10 indexes the queries needed but the schema never created: orders(customer_id), orders(table_id), orders(type), bills(created_at), bills(payment_status, paid_at), bills(customer_id), print_logs(bill_id), loyalty_ledger(bill_id, type), order_items(product_id), customers(created_at). All IF NOT EXISTS, rerun-safe.
  • Every date filter that wrapped the column in date() — which kept the created_at / paid_at indexes useless — is now a half-open range col >= ? AND col < ?. Eight report endpoints, the orders list, the bills list, the cloud-sync heartbeat.
  • The customers page ran 4 correlated subqueries per customer against orders with no index on customer_id. Rewrote as a 3-CTE LEFT JOIN that hits the new index once per aggregate.
  • GET /orders on the orders page runs ~300 prepared calls per 50-order poll (items + addons + table + customer + bill + loyalty per order). Same on the single-detail endpoint GET /orders/:id, which fired 6 prepared calls per click. Both routes now share batchHydrateOrders, one IN() query per relation.
  • GET /orders now defaults per_page=50 (was unbounded), accepts start_date / end_date / before_id for cursor pagination, and returns nextCursor so the UI can actually reach older orders instead of seeing only the latest 50.
  • Every status != 'cancelled' OR EXISTS(...) scan in the kitchen and KDS routes becomes a CTE with UNION over two index-served branches. The chef polling screen used to do a correlated subquery over every order.
  • now() used to return new Date().toISOString() (with T, Z, milliseconds) into columns whose CREATE TABLE uses CURRENT_TIMESTAMP (space, no ms). Two timestamp formats ended up in the same column and ORDER BY / range over those columns sorted wrong within a day. now() now matches CURRENT_TIMESTAMP.
  • "Today" was date('now') / toISOString().slice(0,10) in UTC, which made the AR/BR demo profile's daily numbers land on the wrong day for ~3 hours every evening and bill numbers flip a day late after 21:00 local. Added todayLocalDate(tz?) and dayBounds(localDate, tz?) using Intl so daily boundaries flip at local midnight in the tenant's timezone.

Same 1-year dataset, before and after, milliseconds (3-run best):

  customers list (full page)  40,294  ->  12.4     ~3,200x
  paymentMethodBreakdown       49.2   ->   0.3        164x
  summary orders               13.3   ->   0.1        133x
  summary bills                11.1   ->   0.1        111x
  daily-stats salesToday       11.0   ->   0.1        110x
  orders list "today" filter   10.8   ->   0.1        108x
  reports/tables today per t.  68.4   ->   2.7         25x
  topProducts 30d              59.6   ->   9.0          7x
  sales 1yr daily group        25.8   ->  12.6          2x
  print-history single bill    1.9    ->   0.0
  ledger sum by bill_id        2.1    ->   0.1

At 3 years, the customers list drops from a frozen ~174s to roughly 30ms. The other queries scale the same way since they're index- or batch-driven now.

Follow-ups not in this PR:

  • The frontend orders page still polls per_page=50 every 10s and doesn't read nextCursor. It works but doesn't actually use the new pagination yet.
  • The /bills/:id/print-history call fires once per bill on initial load (the DB roundtrips are gone but the per-bill HTTP roundtrip stays). A batched POST would fix it.

@carvalab
carvalab requested a review from itsbkm as a code owner August 1, 2026 01:47
@carvalab
carvalab marked this pull request as draft August 1, 2026 02:52
@carvalab
carvalab force-pushed the find-issue-performance branch 2 times, most recently from 947a9fe to 26300e9 Compare August 1, 2026 21:37
…oses FreeOpenSourcePOS#208)

Database performance for 100k+ order installs:
- Migration v44 (renumbered past upstream v40-43): adds 11 indexes and normalizes every legacy ISO timestamp to the DB's UTC space form, including order_items.voided_at and the cloud/support outbox tables.
- now() emits space-form timestamps; parseDbTimestamp/utcDayBounds/utcTodayDate centralize UTC parsing and day-range queries.
- Query rewrites: KDS/kitchen active-orders UNION instead of OR EXISTS, customers CTE instead of 4 correlated subqueries, batched items/addons hydration (list + GET /orders/:id), range filters on new indexes, default page-size caps, orders cursor.
- Space-form bounds for whatsapp rate limit, cloud retries, KDS pairing expiry, voided-item cutoffs; UTC-aware timestamp display in the frontend (useFormatDate, printer format-date, product images, whatsapp timeline, tax config panel).
@carvalab
carvalab force-pushed the find-issue-performance branch from 43ccd58 to c1ddebf Compare August 3, 2026 02:43
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

perf: queries and indexes break down at ~100k orders (1yr, 300 orders/day); customers page freezes the app for 40s+

1 participant