Skip to content

April 2022 meeting notes

Asynchronous Discussions

2022-04-13

  • Topic: New Team Workflow
  • Summary: We’ll be experimenting with splitting up into smaller teams and running a 3 week “cycle” from Apr 18 to May 6. This meeting is to discuss workflow changes and communication norms.
  • Participants: Brent, Dominykas, Ghislaine, Kriti, Mukesh, Pavish, Sean

Helpful reading

Our new workflow is inspired by the way Basecamp works.

Meeting structure

  • We’re going to stick to the scheduled time for each topic (agenda and times have been omitted in notes for brevity)
  • Kriti will run the meeting
  • We need volunteers to:
    • take notes - Brent
    • keep time - Sean

Our current problems

  • Our pace is too slow, we’re not completing features as quickly as we could.
    • We need to release soon and get Mathesar into real users’ hands
  • All our discussions involve too many people
    • We need to make decisions more quickly
  • We’re spending too much time debating small decisions
    • Not even sure if some results will matter, our decisions are temporary
    • User feedback will challenge many of our assumptions
  • Kriti is a bottleneck
    • People should be empowered to make decisions independently

The new plan

To address these problems, we’re going to try a 3 week work cycle starting next Monday.

  • From Apr 18 to May 6.
  • We’ll split into smaller teams, each with a very specific goal.
  • You do not need to worry about the big picture during cycles, only your team’s goals for that cycle.
  • After the cycle, we will have a cooldown period of 1-2 weeks to plan next steps
    • Cooldowns are for pausing and assessing the big picture
    • Cooldown reduces discussion during cycle
  • If it works for us, we’ll continue with working in cycles with cooldowns in between
    • Our eventual goal: 6-week cycles with 2-week cooldowns

Team responsibilities

  • Each team will be assigned a project, either a set of features or one large feature.
  • Teams will be small (~2 members)
  • Teams are empowered to make whatever decisions are needed to complete their project.

Goals

  • Team members do not have to do all the work related to their project, but the team is responsible for making sure it gets done.
    • This may change once we have a larger codebase where projects can be less tightly coupled.
  • Team members should use completing all functionality in the design specs in the frontend as the success criteria for their goal unless otherwise explicitly decided

Communication

  • As a rule of thumb, team members should be communicating:
    • frequently between themselves
    • rarely outside the team
  • Each team should decide how best to communicate internally (async, frequent calls, somewhere in between)
  • Please do talk to members of other teams if:
    • You’re both working on the same parts of the codebase and need to coordinate
    • You’re considering not following established API or code standards
    • You’re making product/design decisions that will be difficult-to-undo
    • You need specialized knowledge that would take you a long time to acquire but someone else has at the tip of their tongue (e.g. someone who is familiar with particular parts of the codebase)
  • If you are talking to members of other teams, keep the number of people involved as small as possible to facilitate quicker conversations and decisions

The Upcoming Cycle

Now that we’ve gone over the concept of cycles generally, it’s time to plan for this specific cycle.

Goals

  • Product goals:
    • Finish the next two feature milestones:
      • Initial Data Types
      • Links Between Tables
    • Have enough clarity to plan out the final pre-launch milestones during the next cooldown:
      • Queries and Views
      • UI Styling
      • Deployment
  • Process goals:
    • Reducing product scope to fit into the cycle time
      • Example: If the date picker is going to take a week to implement, then we ship without the date picker. You’re empowered to make the decision to cut it.
    • Reduce context switching during cycles
    • Communication norm changes
      • Discuss in smaller groups
      • Make decisions quickly

Teams

We will split into four teams, with the following team goals:

  • Sean & Mukesh
    • Data types - Number
    • Data types - Money
    • Data types - URI
    • Data types - Email
    • Data types - (loose ends)
    • Foreign keys
  • Pavish & Dominykas
    • Data types - Boolean
    • Data types - Duration
    • Data types - Date / Time
    • Data types - Grouping
    • Data types - Filtering
    • Data types - Default values
  • Brent & Ghislaine
    • Complete Queries and Views design prototypes and specs
    • Complete Queries and Views backend technical spec
  • Kriti
    • Come up with use cases and personas for Mathesar
    • General product plan for UI Styling & Deployment milestones, informed by use cases and personas

Cooldown

The plan for our first cooldown is to plan out the next cycle, including:

  • holding a retrospective on the first cycle went
    • check in on the concerns and distractions we discussed in this meeting
    • we’ll decide what processes to keep and change for the next cycle
  • reviewing product and design work generated during the first cycle
    • Queries & Views design and backend spec
    • product spec for UI Styling & Deployment
  • finalizing implementation plan for Queries and Views
  • figuring out next steps for UI Styling and Deployment milestones
  • figuring out the teams, goals, and timeline for the next cycle

Potential problems

In this section, we brainstormed potential obstacles to reaching our goals for the cycle and talked through solutions. (Private) brainstorming document.

Distractions

  • Eliminated entirely
    • GSoC draft proposal review
      • Final submission date is Apr 18
    • Comms Assignee
    • Weekly planning via GitHub discussions
  • Will only happen during cooldown period
    • Product and design review
    • Team events
    • Public speaking workshop
  • Changes to existing processes:
    • GitHub Issues
      • We will only make GitHub issues for issues that we’re not immediately working on.
        • We don’t need to spec out issues for core team work.
      • Feel free to use GH issues to get things out of your head.
      • If it’s a low priority and we don’t want to deal with reviewing PRs for it, we will mark the issue as status: draft
    • Troubleshooting E2E test failures
      • Feel free to disable brittle tests and create issues instead of spending time on them.
  • No changes, but should naturally reduce with the new workflow:
    • Matrix usage
    • GitHub Discussions usage
    • Hard to keep track of new GitHub Issues
    • Premature documentation / speccing
    • Adding more tickets which aren’t an immediate concern/premature
  • No changes:
    • Hiring & interviews
    • Code review process (for both core team and contributors)
      • We discussed eliminating code review for the core team, but some team members objected.
    • Weekly meeting on Wednesdays
    • Standups
    • API Spec reviews
    • Picking final GSoC contributors
    • Student Design Projects
  • Distractions that were tabled to address later:
    • Increasing scope of design issues / unexpected scenarios
    • Having to change spec during implementation
      • Make it work in the short-term, file a bug for long-term issues
    • Keeping the roadmap and milestones updated
    • Coordinating schedules to find a time for a sync discussion
    • Unexpected Bugs with existing features

A word about GSoC:

  • GSoC is lots of work, but busiest time is almost over
    • This is the irritating period, benefits are later
      • Develop long-term contributors
      • Get word out

Concerns

  • Visibility of internal work might suffer, we’re open source
    • We don’t get any community engagement in our day-to-day work discussions
    • It might actually be better to be intentional about what we release to the community
  • A designer on each team might help unblock work and move faster
    • We’ll figure that out after we see how this cycle goes
  • How do we make sure we estimating scope for a sprint correctly?
    • We’re not trying to make the sprint fit the scope, we’re trying to make the scope fit the sprint
    • This is an experimental cycle, we will iterate on the planning process

Tabled for later discussion:

  • User involvement / research
  • Group chat v small group and direct chat

New tooling discussion

There was a proposal to replace the communication/organization tools used by the core team with Basecamp. We would use GitHub and Matrix only for community-related work.

Benefits:

  • Centralizing project-related comms and files
    • Design specs could be associated with projects
  • Centralizing most relevant notifications
  • Automating standups
  • Tracking non-code todo items (currently tracked ad-hoc via Matrix)
  • Hill charts for progress tracking
  • Ability to make projects public to share them with the community

Concerns:

  • Community visibility
  • Yet another tool to checl

Decision: Table the conversation until after the first cycle.

Next steps and action items

We discussed a lot of things! Here are the key changes that will be happening:

  • The cycle starts on Monday
  • Priorities for this week:
    • GSoC draft proposal review
    • Hiring preparation
      • Localization engineer job description
      • Interview questions
    • Wrapping up current work that isn’t related to your priorities
    • Rearranging scheduled events as needed (e.g. team event)
  • Team organization:
    • Teams should meet and figure out a plan for how to meet their goals.
    • This can happen either this week or early next week.
  • Distraction tracking:
    • Expected distractions during the cycle should be minimized to:
      • Code review
      • Internal discussion needed to pick GSoC contributors
      • Interviews
    • If any other distractions occur, note them for later review!
  • Process changes:
    • No more weekly planning discussion
  • No more comms assignee

Appendix: pre-meeting notes

This section preserves notes added prior to the meeting.

Pavish’s concerns

First cycle teams and goals I’ve been thinking over this the past couple days. We are often overestimating our capacity and underestimating tasks. I’ve been guilty of doing this myself. I don’t want us repeating it for this new workflow. - We have planned to complete all data types in the 3 weeks sprint. - I have about 6 features to complete in this sprint. - Here’s how I can see the scenarios for just one feature - Date & Time types: - We need to search and come up with a datepicker component that suits our usecases. - The analysis itself might take a day. A rough early analysis: - The most suitable ones are jquery based. - A lot of them aren’t even supported anymore. - Jquery itself is about 90kb in size minified. - These date pickers come around 60kb to 300kb. - For comparison, our entire application is around 420kb (When compressed: 100kb). - These would bloat our build size up, and there will be a tradeoff to consider. - Even the most leanest library comes around 50kb. Eg., Flatpickr, the best option I’ve found so far. - Most of these tie themselves to the input (including flatpickr) and perform their own validations. We cannot extend them, say for eg., entering ‘yesterday’ and treating it as a valid value, without forking these libraries. - A few years ago, I spent a week trying to salvage existing date range pickers for a project I was working on, and finally ended up writing my own from scratch. Provided, that project required several specific usecases for the date range pickers. - There is a good chance we may have to build our own components. If we can use an existing library, I’d be very happy. - I see this process (use an existing library or write our own) taking anywhere from 2 to 4 days with focused coding time. - After the lower order components are ready, we need to implement each type: date, time, datetime. - There are some issues here that I see now. We may have more while implementing: - The design spec has separate date and time formats for dateTime. - While implementing the backend, I remember we decided to have a single ‘format’ display option to give user maximum flexibility in the future. - This may lead to a few back and forth between design, backend and frontend before we decide on what we actually are going to implement. - Figuring out the cell input UX and interactions are left entirely to the developer, which will take additional time. - Resolving all these and implementing this might take upto another 2 to 3 days. - Overall, with fully focused time we’ll need anywhere between 4 to 7 days to complete the date & time types. - Considering that we cannot always have that focused time due to other context switches such as GSoC, PR reviews, discussions, personal life stuff etc., it may extend to around 5 to 10 days. - Our whole sprint is only for 3 weeks, which is 3 * 5 working days = 15 days. * This is just one example, I can see FK feature panning out to be more complex than we estimate at the moment. * I think we might need to plan out the tasks before starting the sprint including analyzing each feature with a little depth, so that we can estimate how many features we can actually complete during the sprint.

Design process hurdles * I’ve been noticing issues that come up during implementation of a design, sometimes late in the implementation process. * For eg., Only while implementing display options, I noticed that we need to validate both database and display options before hitting Save. * While this may seem like evident now, it wasn’t during the design stage or even during implementation of database options form. * This lead to having to detach form logic from component (thereby losing the out-of-the-box features provided by the dom & framework), and implement validation for it. * While this task isn’t complex, it still takes additional time, and caused UX issues to consider, like how do I show the user there’s an error when the form’s not visible? * This leads to having to come up with a design improvisation during development, which takes time on it’s own. * These kind of scenarios lead to more conversations and debates on what is seemingly a small change in the end. * Some of the specs are not detailed enough. * For eg., The date and time selector cell interaction is not detailed. * This again leads to the developer improvising design during implementation phase, or a whole new cycle of design to development with a lot of discussions. Both of these take considerable time. * Here are some quotes from the book linked in the parent discussion: Shape Up. * > Wireframes too concrete > making the interface just so can require solving hidden complexities and implementation details that weren’t visible in the mockup. When the scope isn’t variable, the team can’t reconsider a design decision that is turning out to cost more than it’s worth. * > Words are too abstract: > “Build a calendar view” or “add group notifications” sound sensible, but what exactly do they entail? Team members don’t have enough information to make trade-offs. They don’t know what to include or leave out. A programmer who worked in that situation said: You’re solving a problem with no context. You have to be a mind reader. It’s like: “we’ll know it when we see it.”

My suggestion * Each team needs a designer. Right now, there are 4 teams with 2 teams coding in parallel. The teams comprise of one frontend and one backend developer, they also need a designer. * Whenever a design problem is faced during implementation, that designer is responsible for fixing it. * We need to hire atleast one more full time designer. Ghislaine has a lot of parallel work and context switching is going to be hard for her.

Use hill charts with our current workflow * I love the idea of using hill charts. * I often raise multiple PRs related to a single issue, which includes a lot of groundwork. * While I work on an issue which requires groundwork, I’d be the one implementing that groundwork and the issue later. I do not like taking additional time in creating issues for said pre-requisites. * The only major value of it is for everyone in the team to be upto date on what I’m doing. I assume the standups and weekly meetings sufficient for that. * I see a lot of value in using hill charts, so that we can also track progress in such cases more efficiently.

Brent’s concerns

  • I think we can avoid converstion within ourselves, but it’s a bit difficult to avoid with external community members. We should think about how to mitigate this.
  • To me, the most valuable part of this, more than the deadlines or specific goals, is the focus on smaller-group communication. This is the part I personally want to focus on for our initial trial of this style, and where I’m focusing my thoughts here. Concerns with that part of things:
    • I think public visibility is crucial for open source.
    • I think the volume of public discussion right now actually reduces visibility of signal (in all the noise).
    • I don’t know how to make the parts we need to be visible to the public and other internal teammates visible without documenting after-the-fact.
      • ‘watch-only’ rooms for each smaller team? I.e., each smaller team communicates in a way that invites others to view but not participate. May be unworkable.
  • Don’t deal with in this cycle

Dominykas’ concerns

  • I agree wish Pavish about examining first cycle goals in detail to better estimate how long they will take;
  • We’ve not been shy about doing long-form write ups, which is good;
    • but I’ve been often disappointed with the quality of my own writing;
    • in my experience, I’d say that the longer my write-up, the more likely I am to be misunderstood;
      • that’s partly expected, because long explanations tend to introduce something new;
      • at the same time, I’m convinced that my writing skills (which has a lot of overlap with thinking skills) could be significantly improved;
    • I would be interested in honing my technical writing skills;
    • I would be interested in starting a writing club or something similar with that objective.
  • I support using hill charts, and I’m super interested in what other tools we can use to improve our workflow;

  • I like Basecamp’s philosophy and approach;

    • I can see why their articles were linked many times in this doc.

  • Cycle isn’t about estimating how long it takes to build, it’s about figuring out what we can build in that timeframe

  • Transition period; experimental