Documentation in Data/Analytics Project: An underestimated ally!

Documentation is something that gets misunderstood a lot IMO.
Sharing my thoughts on the topic (applies to Data and beyond)

Its Objective: Context & Clarity

In more details, you want to make sure that the future readers/users (yourself included!) of the model/transformations/query/code/etc will understand what has been done without the need to come back to you and ask you to understand why and what you did before.

Its Flavours: More diverse that what we might think

A couple of ones I consciously consider:

  • Code -> Your code should be as self-explanatory as possible. Write clear simple code, not a clever one!
  • Naming -> helpful intuitive naming (for files, datasets, data models, columns…)
  • Comments -> Additional comments in the code (if a comment is confusing or too complex/long, it backfires)
  • Testing -> Testing helps the user understand the structure and data of the model (like uniqueness testing help understanding the granularity)
  • Documentation files in codebase (as needed) -> if the naming makes things self-explanatory, this becomes the place for more context, like business context
  • Documentation outside codebase (like in confluence) -> Mostly for more complex contextual subjects like documenting the business use case of an ML model, reporting pipeline, architecture, complex data source, etc…

Ultimately, one needs to I ask oneself:

Will the next reader face friction with what I am creating ?

If the answer is yes, simplify further and document better!

Head of Data & Analytics @ Fabulous. On Data, Analytics, Tech, Business and Life…

Love podcasts or audiobooks? Learn on the go with our new app.

Recommended from Medium

WANT SOME

Board Games That Help Software Engineers Hone Their Craft

Lambdas (Java 8)

I know what a Tax Audit is, but a tech audit?! Don’t panic… just read

I know what a Tax Audit is, but a tech audit?! Don’t panic… just read

Implementing Clean Code in Our App, Why?

READ/DOWNLOAD%! Introduction to Recursive Programming FULL BOOK PDF & FULL AUDIOBOOK

3 Reasons Why Every Kid Should Go PRO in Coding

Check out how a hybrid BMW looks in your own garage

Get the Medium app

A button that says 'Download on the App Store', and if clicked it will lead you to the iOS App store
A button that says 'Get it on, Google Play', and if clicked it will lead you to the Google Play store
Ahmed Omrane

Ahmed Omrane

Head of Data & Analytics @ Fabulous. On Data, Analytics, Tech, Business and Life…

More from Medium

A Collaboration Cycle (Part I: Set-Up)

Overcoming reproducibility challenges in model validation

Lessons learned automating Snowplow schema management

Snowplow plowing snow off the road

Creating High-Quality Staging Data with a NoSQL Data Migration System