add best functions by name page - #2061
Conversation
|
Need to look into this, but feel like it's potentially an expensive query (if it's not based on family) that perhaps could be something we do based on a dump of the database, or some daily batch process or something? |
|
It forces selecting a platform and preset to avoid a full scan. I ran EXPLAIN ANALYZE and saw it used the preset_id index. On my system (old 2020 macbook) it was about 5ms to do the three queries on 398 rows. I'm not sure what preset has the most scratches but I see Rhythm Tengoku has more than average at 4241. I tested that and it took about 15ms. I could turn up the caching if it seems worthwhile. |
| @method_decorator(globally_cacheable(max_age=5, stale_while_revalidate=1)) | ||
| @action(detail=False, methods=["GET"], url_path="best-by-name") | ||
| def best_by_name(self, request: Request) -> Response: | ||
| if not request.query_params.get("platform") or not request.query_params.get( |
There was a problem hiding this comment.
preset dictates platform though, why are both needed?
There was a problem hiding this comment.
You're right, revised this to remove platform
| def _best_by_name_eligible_queryset(self, request: Request) -> QuerySet[Scratch]: | ||
| queryset = ( | ||
| Scratch.objects.all() | ||
| .exclude(name=UNTITLED_SCRATCH_NAME) |
There was a problem hiding this comment.
why? doesnt this just add an extra clause to the search (i.e. its more unhelpful than helpful)?
There was a problem hiding this comment.
My original reasoning is that the Untitleds are random unrelated scratches so getting a ranking of % complete doesn't make sense. I checked this with a synthetic dataset of 100,000 scratches locally with 10% being Untitleds and I got:
216ms with Untitled exclusion
231ms without Untitled exclusion
I'm not sure what the real ratio is like on the site
|
I'm still not super keen on the general concept of this functionality.. but will bow to other maintainers opinion. I feel like we should be able to fix your cloudflare issue easily enough (we have other known users using the api -> reach out to us on the discord). |
I most recently asked about this on August 1 but didn't get a response https://discord.com/channels/897066363951128586/897075447534354462/1532938403036004665 |
2 participants
This puts together a "best scratches by function name page. The use case is:
On the projects I work on, people tend to get a function to 90%+ then get stuck, and people forget about the scratch. I want to be able to go back and see these scratches. My function_finder tool used to work for this case by using the API but now with CloudFlare it's non-functional. https://github.com/Xeeynamo/sotn-decomp/blob/master/tools/function_finder/function_finder_psx.py
This adds:
GET /api/scratch/best-by-namegroups scratches by function name, ranks by match %, filterable by platform/preset/depth/min-match/search/scratches/best?platform=saturn&preset=<id>page consuming the apiScreenshot with fake data: