As organizations grow and evolve, they need to adapt to the challenges of managing more and more people and respond to the changing landscape they operate in. However, growing the team without keeping developer satisfaction in mind can lead to long-time staff members becoming jaded and burned out. This leads to a high turnover rate as well as valuable knowledge becoming unavailable over time.
Recently, we were approached by a leading European online travel agency who recognized these symptoms so common to scale-ups. Initially, they hired us to advise them on technical and architectural questions, but we ended up providing organizational and engineering process-related consultation too, which helped them improve their development processes and get rid of some of their unrecognized communication problems.
In the following case study we outline the work done by RisingStack, with the intention of helping IT decision makers through sharing our experiences.
Architectural Consulting
The first part of our assignment centered around the introduction of microservices as well as common modern practices such as automated testing improvements, CI/CD workflow, code reviews, and such.
Whenever it comes to fundamentally overhauling an architecture, we need to look deeper to see what problems lurk behind these proposed changes and whether undertaking this shift is achievable. To uncover issues that might hinder our plans, we agreed with our client to thoroughly investigate the situation by:
- Getting a full picture on the technical aspects of day-to-day development: The codebase, the development cycle, and the deployment process.
- Understanding the organization and the flow of information within it:
- How are the long-term development roadmaps created?
- How are they translated into epics and stories?
- What is the flow of information from where the idea of a new feature or general functionality is conceived to the actual development of it?
- How do developed features get accepted?
First, we assessed the current state of the codebase and deployments. Both of them were generally well thought out, but two things stood out:
- The majority of lines changed in the past couple of years belonged to a handful of people despite the organization employing around 20 developers.
- A lot of code hasn’t been touched for years, while other parts of the code essentially solved the same problem that has been worked on in the recent past.
The first point hinted at the heavy reliance on silos in the organization, while the second one showed symptoms of poor communication.
Understanding How the Organization Works
Next, we started conducting interviews with all related parties, beginning with the C-level executives and directors. We needed to get a sense of the organizational structures we were dealing with. According to them, the company was split up into 5 departments.
- Finance
- Sales / Business Development
- Marketing
- Development
- Customer service
As all five departments could submit development needs, they maintained the product’s roadmap together. Based on the roadmap, Product Owners supervised the planning of new features and functionalities. The plans were mostly created by the UX team. These were chunked up into development projects carried out by the development teams using the agile methodology. Teams were also aided by a Project Manager whose job was to ensure the development was progressing according to the plan. Or at least that was the shared belief of how the organization works.
However, the nicely drawn organizational diagrams seldom reflect the truth on the day-to-day reality of teamwork. To discover the actual, informal organizational structure we conducted interviews with every team, every single person within the company who had anything to do with the product. Beyond C-level execs and directors, we also interviewed product and project managers, tech leads, developers, QA engineers, and manual testers.
Uncovering the Informal Flow of Knowledge
During these interviews we were able to understand the history of this company’s development, and how growth affected their way of conducting business.
Initially the company functioned like a small startup. With only a couple of developers, people could just nudge each other in the office, ask for something to be done, and deliver features. If a request was overheard by another stakeholder, it might have been challenged or deprioritized. The company worked completely informally, yet very efficiently. No lengthy meetings, no unnecessary planning, just people deciding how to spend their time most effectively to achieve common goals. This worked perfectly while there were only a handful of stakeholders in the organization, however, as the company grew, it needed to get organized, and thus emerged the 5 departments. Still, the leaders of said departments could easily reach the developers if they needed something.
Our client’s product had key advantages compared to its Central European competitors, which made it the undisputed leader of the market in this geographical area. These specific features could only be developed by one person at a time, so the other developers were helping out where they could, to make themselves useful in the meantime.
The channels of favors were formalized, resulting in the requests that previously were mere favors becoming mandatory tasks, those who asked for these favors in turn became supervisors to developers.
By 2017 the product was basically done, no development happened on its core, besides marketing campaigns or sales-deal related updates. However, these sales related requests started becoming more and more ambitious, requiring more and more work, thus our client started hiring more and more developers. This was necessary for another reason too: developers started leaving the company, leaving just a handful of old-time developers in the organization, and only 2 of them wrote code actively. The others got appointed into leadership roles. However, more people did not result in faster delivery times, and people had to crunch more and more to meet their targets, which were missed by bigger and bigger margins.
On the business side, as time went by since its founding, our client’s product quickly overgrew its initial national market. It needed to look for other areas in Central Europe. They could maintain growth by acquiring other businesses, thus innovation was not key for a couple of years to their success. However, through these acquisitions, the company’s application portfolio became so fragmented that it became unwieldy to manage: a large overhaul was necessary to unify these acquired products, which led to the need for more organized development.
At this point, in 2019, they hired a consultant to help them rectify their processes and update their organizational structure, but as usual, they only got to the formal level: leaders were consulted, a nice structural diagram was drawn, development teams were created, but the informal channels of favors-later-task-requests remained.
Aligning the formal and informal way of working
While in theory there was a roadmap to follow, and there were people appointed for the supervision of different aspects of the work, the day-to-day reality was pretty far from the formally defined organization. It became clear that we needed to find the necessary missing pieces that prevented the reorganized structure from taking hold.
During the interviews, different people raised seemingly unrelated issues, but at the end three overarching themes emerged:
- The lack of knowledge of the actual application within the company
- The lack of a company-wide definition of done
- The exclusion of developers from setting deadlines
The first was a tricky one: because everyone had some sense of what the application was capable of, it was never stated explicitly, except for newcomers. When we got a tour of the application it was seemingly thorough and complete. However, during group interviews people kept correcting each other on specific edge cases they needed to handle. So while we got a static picture of the application, when we went back asking about specific capabilities, it became clear that the knowledge seems to be lost, mostly because the person responsible for a given functionality has already left.
However, manual testers had a pretty good grasp of both the general functionalities and edge cases too. It has also turned out during the interviews that they maintained a sort of “User Manual” of the product, that was a couple of hundred pages long, and no-one ever read it. Thus, in order for this knowledge to be available during the planning of given functionalities, we suggested that the relevant manual testers should be a part of the planning and design process, so no contradicting and superfluous features would be commissioned.
The second turned out to be the missing keystone of the organizational shift. While each team, and each organizational unit had their own definition of done, there was no company wide one. This way, the development practices and expectations of different actors were completely out of sync with each other. This also meant that developers had initially no easy way of pushing back on those who used to ask for favors or later create tasks for them, as it would have required too much explaining. Not being able to push back on requests resulted in the old ways trumping the organizational structure. We advised all departments and business leaders to agree on a definition of done.
Lastly, while everyone thought that developers were included in the planning of projects, this usually happened after the commitment has been made to partners or clients. Thus resulting in unrealistic expectations towards the developers, and not providing them enough room to make these on-off solutions scalable and maintainable. We advised to update the deal making process, to include consultation with Development before taking on any commitments.
Did it work?
We followed up on the progress 3 months later. While the involvement of testers into planning resulted in some initial pushback from the department heads, they got included into the design process, which almost immediately resulted in better alignment between the teams, as finally, testers became officially appointed as “keepers of truth” regarding the product as it is, creating a mental connection for questions that seemed unanswerable before.
To put it simply: no-one ever considered asking the question “how does this actually work now”, besides consulting the codebase, and even if it did, manual testers never came to their minds. This led to better information sharing and collaboration between the teams, as they were freed from the toil of investigating and rediscovering modes of operation of the application.
The definition of done was a more pronounced success, ss the head of development and the development teams have previously agreed on a development flow, which would have been a fairly standard one:
development <-> unit testing <-> code review -> QA -> done
⌃ ⌃
|_______________________________________________________|This previously agreed upon flow was never implemented due to the shortcomings detailed above, but after our consultation, it finally came through.
Taking the time to explain other departments on the process, and having them accept that a feature is not simply done upon the first completion of the code, letting them understand the need for QA made it easier to push back on favors and allowed the organization to take a shape that was closer to the formal one.
However, this would not have been possible without reducing crunches. With bringing developers closer to the actual deals, consulting them on what can be done at the moment easily, and what needs extensive development, while incorporating these investigations and their current development schedule into the deal making process (and the added benefit that they could rely on the knowledge accumulated by manual testers), crunches became rarer, deadlines became more realistic and less often missed and the business side became more satisfied with the development’s work too.
Do you face the same issues?
In case you found yourself heavily nodding through reading our case study, then you hopefully know that you’re not alone with your problems, and that there are ways you can address them. Let us know in the comments about your experiences. In case you’re looking for consultants to solve your issues, feel free to reach out to us.
