A Personal Anecdote
At a past job, a very successful contracted developer became a full-time employee. After a few months, they were frustrated about being less productive as an employee than as a contractor. We had a conversation about the difference in the roles and expectations between contractors and employees, and worked to re-shape the idea of productivity for them. It’s an interaction that stuck with me and which I’ve shared personally a few times.
Contractor Roles
With contractors, we looked for someone to complete an assigned task quickly and correctly. We did not expect contractors to have institutional knowledge, so they depended on the task details provided and had minimal concern about externalities. The work was transactional.
This even created some rivalry between development contractors and the Quality Assurance team. Contractors could justify returning tickets which lacked detail, even if the missing information was “common knowledge” in QA. If a task was insufficiently described, that was a requirements problem, not a development problem, and a ticket returned to the creator did not hurt the contractor’s statistics like a test rejection for an incorrect or incomplete solution.
There was some give-and-take in this process, but overall it was reasonable for a contractor to expect the necessary information be provided for a task. This might seem to be a narrow view of contracting, but it was the expectation we worked with at the time.
A successful contractor solved the provided problem.
Employee Roles
With employees, we expected the accumulation of institutional knowledge and understanding of the broader context of the work. The employee developer needed to understand the business case for the work, and how a task fit into it. They needed to have an idea of the externalities and how decisions affected other processes or projects. An employee was expected to ask, “Why?”.
Further, an employee was often expected to take longer at a given task because of these additional requirements. We asked more of employees, both in terms of knowing the concerns and in taking the time to obtain information or inform stakeholders.
A successful employee solved the underlying problem.
Overlap
Employees could be called on to solve problems as though they were contractors in a pinch, but you could not reliably ask a contractor for the opposite. This further demonstrated the difference in expectations for the roles and that there were different productivity measures for each.
Productivity
So what is productivity? Other than the quality or state of being productive, it really
Lines of code don’t provide a meaningful measure. Number of commits isn’t necessarily representative of someone’s effectiveness. Code coverage doesn’t always provide the guarantees we expect.
This becomes more complex as you look at different roles and the associated goals. A senior developer supporting multiple team members should be expected to have lower individual code output, while the overall team’s output or quality measures improve.
Measures
Depending on the tasks, productivity may track Key Performance Indicators (KPIs) or fit into Objectives and Key Results (OKRs). Or maybe you have different jargon for “things to measure and measuring things”.
Some of these proposed measures are for individuals. Others are for teams or even products. I will not dive into the details of each unless people express interest in discussing them.
Tickets:
- Number of test rejections
- Total defects identified
- “Escaped” defects
- Size of the backlog
- Average ticket age
Coding:
- Necessary changes identified during code review
- Maintainability scores, e.g. cyclomatic or cognitive complexity
- Test validity, e.g. does this cover the business cases
Project Tracking:
- Improved estimate accuracy (usually very difficult)
- Individual or Team Velocity (if adequately implementing agile principles)
- On-time deliveries
- Customer satisfaction
Selecting Measures
The best recommendation I can give is to avoid any measures which you can imagine ways to manipulate.
- Lines of code can be increased with formatting and comments.
- Commits or Pull Requests can be made smaller and more frequent without improving quality or output.
These are two simple ones, but try this exercise with any measurement you might be using. If you can game the system, what is the measure really telling you?
Conclusion & Feedback
Ultimately, I don’t have an answer for how to measure productivity for you or your team. It can depend on many factors, including team and company leadership. Productivity measures may need to align with corporate goals or with management directives, and they can change significantly with the type of work or product you support.
What are your measures of productivity, and how do they vary by role or team?
FAQs
Q: What are some common measures of productivity?
A: Some common measures of productivity include lines of code, number of commits, code coverage, and defect density.
Q: How do you measure productivity for different roles?
A: Productivity measures can vary depending on the role. For example, a senior developer may be expected to have lower individual code output, while a junior developer may be expected to have higher code output.
Q: Can you provide more information on the measures you mentioned?
A: Yes, I can provide more information on the measures I mentioned. Please let me know which ones you would like more information on.
Q: How do you select measures for your team?
A: The best recommendation I can give is to avoid any measures which you can imagine ways to manipulate.
