The collective waste caused by poor documentation

Full-stack web development

• This happens all the time in web development: time-consuming, trivial, silly issues with Webpack, Typescript, React, Next.js, etc.
• I remember spending days trying to deploy Migaku Experiments Platform to AWS Lightsail, and I could not for the life of me get it to work... It was my first time ever creating a backend server (instead of using Firebase for a database + authentication) and deploying it to a bare-bones Ubuntu instance, so I likely wouldn't have as much trouble now, but my point is just that it shouldn't have been that complicated! And yet somehow it was. Neither the documentation nor other external sources online were helpful enough. I eventually gave up and switched to EC2 (which has similar documentation issues, but it's so popular that I was able to follow a tutorial to get things working).
• I remember a few months ago wasting ~5 hours trying to debug vercel/@nft crashing the Next.js build of helper.ai (a project I was contributing to while working at Gumroad). I never figured out the precise cause, but moving a function definition from one file (that I was importing from) into another file (and then importing from the new file) fixed the crash... I don't remember the exact error message off the top of my head, but I remember finding other Github issues complaining about the same exception, with unhelpful suggestions for fixing it...
  • A similar bug happened some time shortly after in a route handler, and the solution was also to rearrange some code to appease Next.js.
  • Looks like people still have issues with that package: Github link
• I spent at least a week (50+ hrs) inhaling Next.js documentation, obscure Github discussions, and insightful blog posts to better understand React server components (RSCs), server actions, caching, and how to best use all of these features. At the end of all of that, I felt like I had a really strong understanding on the topic, and it made building things in Next.js a lot easier (I wrote more about this experience here).
  • And my coworkers agreed! After leaving the company, one of them told me something like "Your Next.js leadership will be missed!", which I was flattered by, and also amused by since I had barely 12 months of Next.js experience by that time! But this is the time investment you have to put in to understand any of these technologies and wade past surface-level discourse online. It shouldn't have to be this way... Developing a surface-level, functional understanding of Next.js doesn't take long, but developing the understanding around RSCs and server actions that's needed to evaluate when/how to use them and whether people's claims online about them are valid takes much longer.
  • "But doesn't mastery inherently take time for some things? Why expect you can become an expert in a day?" I'm not necessarily disagreeing with that... Maybe I'll write about the nuance some other time.

Firebase

• (in hindsight, this isn't a good or clear example. I don't precisely remember what I was struggling with in this situation, but given my current recollection, it sounds like I was just struggling as a beginner and that I simply needed practice thinking about database schemas?)
• Firebase was the first solution I ever used for adding authentication and a database to an application of mine. Hosting my own server + database seemed too complex at the time (this was in ~2018, less than 1yr into my programming career). I didn't know any best practices for key-value/document-based databases. I remember stressing a lot about this passage from the documentation:

  • Because the Firebase Realtime Database allows nesting data up to 32 levels deep, you might be tempted to think that this should be the default structure. However, when you fetch data at a location in your database, you also retrieve all of its child nodes.

    • (the exact passage may have changed since then, but I vaguely remember it looking like this)
  • I can't remember precisely what about this was confusing me. I maybe was just struggling to apply the ideas to my specific project (a meeting attendance tracker). I just remember stressing in the speech and debate room with a pencil-and-paper about how to structure my schema to avoid the "waste" and potential performance issues that the documentation was warning about, and I felt like I had no feedback to know if the schema I settled on was the "best" or if it had issues. I couldn't find or piece together a clear/certain answer from the documentation or other online sources (like Stack Overflow). I spent at least a few days I think banging my head about it before I settled on a schema (and I think that time/effort felt unnecessary or like a waste of time to me, because I thought "surely this is a solved problem... and yet I can't find something that tells me the right answer, and so now I need to potentially waste a lot of time trying to figure it out or trying to act given that lack of knowledge. Would be so much faster if someone just told me the right answer...").
• I later had a similar confusion with MongoDB I think (because Lucas, the CEO of Migaku, was discussing with me whether we should use MongoDB for some project). I vaguely remember reading this idea that if you wanted to query for all "documents" in a collection containing a key-value pair (e.g. all user documents that look like { _id: 123, is_verified: true, ... }), this query would "fetch" the entire document to check that is_verified property (and so therefore, you wanted to be smart about structuring your data to avoid querying excess data? But then I guess there was also the denormalization idea where you want to group data that you plan to query together?). Again these ideas don't confuse me anymore, but as a beginner it felt so hard to get clarity on what felt like relatively simple questions on how to best structure the schema and what types of applications/data are document-based databases well-suited for.

SQL

• I vaguely remember struggling in the early days of learning SQL. I think Lucas (the CEO of Migaku) suggested that we use it for the Migaku Experiment Platform, so I needed to learn it. I didn't have too much trouble learning the basics, but when I wanted to answer certain types of questions using our data + schema, I didn't know what the "best" query would be to do that. I could come up with queries and/or data transformations in code to get the answers I wanted, but I wasn't confident it was the "best" solution (in terms of performance mainly i.e. "Is there a faster, more elegant way to express this as a single SQL query? Or do I have no choice but to do some processing in code after retrieving the query result? Who knows!"), and I couldn't find clear answers online explaining what the best solution would be. And fair enough I guess, since at a certain point, your situation is too specific to get tailored advice from Google searches maybe, but...
• Thoughts like "is this the correct way (and/or a "good" way) to do things? How would I know? I've never worked on a real project at a real company...".

Datascript

• Roam Research uses Datascript for storing graph state, so I had to learn it to create more powerful queries and extensions. I ran into problems similar to the ones I ran into with SQL. I knew in theory how to do everything, but I didn't know with some certainty what the best practices were for certain types of queries or problems.
  • like what WOULD be the best way to filter blocks by attributes given the structure of the data (that I can't modify directly, since it's Roam's database schema)
• I remember having to figure out a lot of things experimentally while working on a Roam Research query language.
  • I vaguely remember figuring out experimentally that executing multiple queries and diffing/transforming those results in code was way faster than trying to construct a single, large query to produce (mostly) the final result.
    • (I guess it makes some sense purely through the lens of latency; part of the reason people may want to avoid multiple database queries is because these queries usually happen over the network, but Datascript is an in-memory database)
  • If I recall, I did a lot of experimentation to come up with this contraption to performantly query Roam Research attributes.

(def attr-values-rule
  '[(attr-values ?block ?attr-eid ?v ?is-single-value ?parse-one-line-attr ?extract-attr-values ?eid->block-refs)
    [?block :attrs/lookup ?attr-block]
    [?attr-block :block/refs ?attr-eid]
    [?attr-block :block/string ?attr-string]
    [(?is-single-value ?attr-string) ?single-value]
    (or-join
     [?attr-block ?attr-eid ?v ?attr-string ?single-value ?parse-one-line-attr ?extract-attr-values ?eid->block-refs]
     ; One-liner attribute
     (and [(true? ?single-value)]
          [(?parse-one-line-attr ?attr-string) [_ ?v]]
          [(?eid->block-refs ?attr-block) ?refs]
          [(?extract-attr-values ?v ?attr-eid ?refs) ?v])
     ; Multi-value attribute
     (and (not [(true? ?single-value)])
          [?attr-block :block/children ?children]
          [?children :block/string ?v]
          ; Ignore empty blocks (even though Roam adds them to :attrs/lookup)
          (not [(re-matches #"^\s*$" ?v)])
          [(?eid->block-refs ?children) ?refs]
          [(?extract-attr-values ?v ?attr-eid ?refs) ?v]))
    [(ground ?v) [?v ...]]])

• I had to do a lot of digging to figure out how to recreate the results from Roam's native {{query}} widget.

• But again, a lot of this work felt like a waste of time - if only I could just ASK Nikita (nikitonsky) or someone else how to do certain queries optimally (without needing to run benchmarks myself to re-derive best practices) or have these topics more clearly described in the documentation.

• Some questions/topics I couldn't get clarity on (based on notes I found scattered in my Roam graph)

  • Does Datascript re-run assignment clauses of constants for every block in DB? (e.g. defining a regex inline with re-pattern vs passing the object in as a query input)
  • How do you even debug Datascript queries and get visibility into them?