Developer Diary 3: End of the year, end of v0.3.0 development, User Acceptance Test Success
As we end this development cycle, and we look at how well we’ve done at achieving our goals, our Technical Director, Brian Ballsun-Stanton, reflects on the year.
While my first dev diary has our expensive sentence, the way I framed our goals when talking about feature priority was, “the thinnest possible bridge between notebook creation and export.” We need to be able to run demos of this software in the new year, and by being able to take people on a journey from notebook creation, to data collection, to (maybe) round trip, to export — we can exercise all aspects of the system. This lets us avoid the trap of “too many features” in our data collection while still requiring manual design of data collection steps. That bridge also would need to try to support our CSIRO Geochemistry water sampling workflow. I had our devs focus on water sampling because of the multiple pH/Eh samples required in that workflow. The rest of the module is implied if we get water sampling right.
To a first approximation we achieved this goal. Follow me on the journey across this incredibly narrow bridge.
Hark! It says… "Create Notebook!”
On 8 December 2021 I built a simple notebook without help from our devs and without need for manual intervention. I was able to specify metadata, form and section ordering and fields, annotation and certainty labels and got a bunch of JSON. Once loaded onto CouchDB1 I loaded the app in Chrome and saw:
A short setting the auto-incrementing2 values and new record later, and:
The form and section and fields and field metadata I specified was there for (me) to see! It worked! After making sure the data synced, I was able to run my exporter against the CouchDB instance and see:
A folder with a CSV in it that had my data in the csv! (It’s very important to check that final step.) While there is plenty of distance to go on export3 — I was able to see the data that I had entered on the server in the form that I had designed by GUI! THINNEST POSSIBLE BRIDGE, ACHIEVED!
Some other neat stuff
We encode a visual model of form relationships (and also implement them) so that both containing and linking relationships are supported and can be specified using the GUI4. We also have map widgets from Steve, an entire external Authentication feature-set, merge-conflict detection5, multiple-sections, and human-readable identifier search. All of this was developed during the extremely miserable Sydney lockdown and our developers over at AAO deserve a big round of applause.
Reach out to us at [email protected] if you want to be involved in our private demos early in 2022.
User Acceptance Test Reports
After a nice long chat with Nathan Reid of CSIRO Mineral Resources, we demoed all of the work that AAO had put in this development season. He was able to use the notebook functionality, export data, and even create a notebook. While there were plenty of bugs (most of which we already knew about) — there were no showstopping problems and everyone agrees that we will be able to use FAIMS3 to collect data in the field sometime next year.
Which is nice.
With this v0.3.0 User Acceptance Test success, we’re calling 20216 to a close for development work and FAIMS3. We already have way too much planned for January, but we’ll release news of that in the new year. 2022 will be a big one.
Reading this week
(Insert appropriate exhausted look here — but happily I’ve got Newsblur as my RSS reader and can just look at what stories I shared in the last two weeks)
Patrick McKenzie on The Fraud Supply Chain
Scharon Harding’s writeup of someone restoring a keyboard and trackball from an old ICBM control panel.
Rachel on ways a (code) builder culture can fail
Apenwarr on sociotechnical patterns from our current software development trends.
Brian Krebs on how the net is held together by spit and bailing wire.
Chiptunes of Sea Shanties (Youtube Playlist, autoplay) by The Longest Johns and Lucy Humphris
More automated approaches are planned — but this was something that I could write a quick python script for to move between “draft” mode and “multi-user” mode and wasn’t part of the critical path for the “thinnest possible bridge.” We need a moderation step so that we can control who is collecting data and more importantly sharing files with our app, from both a sustainability point of view and from a compliance with mobile app store policies. If people are running their own servers, they will be able to do this moderation for themselves.
We plan to support only local-device simple auto-incrementers for our primary release. While we experimented with all sorts of ways of uniquely identifying records (for humans, mind you, uuids exist for the computers) in FAIMS2, local-to-device increment-by-one auto-incrementers sufficed the vast majority of the time and caused the least performance implications. More complex incrementer-plugins can be commissioned from 2023.
Making attachments work, figuring out the best way to rename photos, making the geospatial data export in some sane fashion, figuring out how to map multi-valued attributes into a CSV, testing the stuff, authentication, etc…
I’ve waited 10 years to say this. I’m not going to stop repeating it in this blog post.
Resolution is TBD… but we can detect them!
I still think that April 2020 has been a very very very very very long month, but other people tell me that we’re in December 2021 so I should probably believe them.
For more news, subscribe!
