Balancing Performance with Code Readability

This is a topic much discussed in the industry and the position I take is a common one: Code first for readability and support, refactor for performance as necessary. There are notable exceptions to the wisdom of taking a readability-first approach. In general you will know if your code is one of these exceptions. It could be graphics-intensive operations, operations widely used and distributed in libraries, and similar situations where you know critical paths through the code a priori. For the 90% case, where squeaking out every nanosecond of performance is not critical I like to use the following steps:

1. Write code to be as readable and supportable as possible
2. Validate that the performance is acceptable
3. If performance is not acceptable, profile to find the hot spots
4. Identify techniques for improving performance of these hot spots
5. Using the Strategy Pattern
1. Extract each module to  be refactored into a separate component and introduce a simple factory to serve up the existing implementation
2. Introduce a higher performing implementation and update the factory to use this new strategy
3. Add comments as necessary to the new strategy implementation

The factory gives us much more flexibility. It allows us to restore the original strategy if we later find that we really don't gain enough overall performance to warrant the new version. It also allows us to experiment with different strategies for improving the performance before deciding on one.

I am of the school of thought that a lot of comments in code is a design smell that the code wasn't as well thought out and decomposed as it should be. I think that most code should be written to be self-documenting with appropriately named methods with small scope. There are certain exceptions to this. Writing for performance is a common way of adding unintentional complexity. Often this added complexity is unavoidable. We should add sufficient comments to explain the code and the reasoning behind the extra complexity so that someone further down the road does not reverse our decisions (perhaps for readability) without understanding the implications of that reversal.

Next week I will push into a use-case where I opted for readability over performance in conflict with published performance recommendations from the platform vendor.

Effective Code Reviews: Case Study

As mentioned in previous posts, mentoring is an important and rewarding aspect to software leadership. One of the more effective ways to teach is via thought-out code review feedback. Performing code reviews is also an excellent way for the reviewer to learn and pick up new tricks. I often find it helpful to be on each end of the code review.

I solicit code reviews of my work when coding something particularly mission-critical and/or complex and encourage the same from other members of the team. There are many ways to run code reviews and perhaps I will discuss this in a future topic. Code reviews can involve a team or be one-on-one. The case study discussed here was a one-on-one case where I was the reviewer. Usually this has worked well for me. This situation had some unique challenges, however. Specifics of this situation:

• Working with a senior, talented, open-minded developer
• Developer is a non-native English speaker
• Developer is very open to feedback
• Developer's style resulted in overly complex code
• Developer frequently requested reviews
• Most of the request reviews span numerous modules

Problems this posed for me:

• Often times the code was just too complex for me to perform an effective review and root out potential side effects
• The language barrier reflected itself in the code in the naming of methods and variables
• The time demands of the frequent requests were too high

To address the time constraints, I applied a common technique, which was to ask to limit the scope to areas of highest concern. This did not work, as the developer was not able to identify specific areas. The inability to identify "chunks" that could be easily identified to treat in isolation was another symptom of the complexity.

This situation continued over a period of weeks. The bulk of my feedback centered on addressing the complexity issues and was in the form of email. However, each time I did this I felt I did not grasp the nuances of the changes well enough to provide adequate feedback. I found that the developer would make the specific requested changes and then the next day make the same complexity mistakes in the next piece of code that he submitted. To make matters worse, we were in a critical part of the project where time pressures were significant.

The real fix is to fully refactor the functionality. This is something we scheduled, however, it was not realistic in the short-term, so it was important to stabilize the code as much as possible. Equally important was my desire to help this talented developer with his complexity issues so that the problems would not be repeated in future projects.

Finally I decided that, despite the time-zone challenges and the additional time demands, the most effective way to perform reviews in this situation was to do them interactively with the developer. While voice is normally my first choice, in this situation I found that interactive IM sessions were more effective. There are two reasons for this:

1. It gave both parties an opportunity to think carefully before each reply
• From the developer's side this extra time gave him the opportunity to translate as necessary. 
• From my side it gave me time to analyze and provide recommendations as we went through the code.
2. It provided a written transcription for reference during and after the discussion. 

My biggest hurdles were understanding the motivations and side effects of each of the changes. So the review then became something like this:

• Me: Method X, lines Y-Z: What are you doing here and why?
• Dev: Explanation
• Me: Did you consider factors A and B?
• Dev: Actually, I think there may be a bug C 

( One of the things I like best about code reviews, whether I'm reviewing or being reviewed is that in discussing the code often the coder himself will stumble on issues)

• Me: What if you took this 4-line return statement and simplified it to a method call

etc.

The end result was that I felt I was finally able to provide effective feedback and identify specifics of how to reduce the complexity of the code.

The lesson learned here for me is to not wait so long to consider alternate communication mechanisms if I find one mechanism is not working well. This is a lesson I try to apply to many other aspects of communications.

Key traits of a software leader - Dedication, Love for Mentoring, Balance and Perspective

• Dedication

Leaders and senior developers often know what to do, without the need for external direction. Of course you must do what your manager asks of  you, but once you start on it, you should usually be able to complete the task without requiring management oversight. Are the requirements clear and complete? If not, find out who to talk to get them clear and complete. If you can't get clear requirements from the prodouct owner for this task, come up with your own clear and complete requirements (written or in the form of tests) and then give these to the product manager or manager as a starting point. Hold a meeting with the stakeholders and leave with the requirements clarified. Take a similar approach for every other aspect of the delivery. Push for answers you need, make proposals along the way, always get buyin from others of the team, and don't be afraid to hold meetings when email, issue tracking systems, or other communication mechanisms fail you. Ensure that requirements end up in a written form that is signed off on by all stakeholders.
Whether it's the near-term objectives of a sprint or longer-term objectives, a leader is strongly invested both in his/her own commitments and in the teams' commitments. Are others on the team in need of your help? How can you ensure the success of the entire team? Like everyone on the team, leaders will put in extra hours during legitimate "crunch" times to meet important milestones. This does not mean that you should be expected to work 80 hour weeks ad infinitum. Learn when it is truly important to put in the extra hours and when it is unreasonable. If you are like many developers, you enjoy your work enough to want to put in longer hours. Instead of putting in an 80 hour week for your current sprint, consider spending time learning on your own and/or playing around with new technologies if not in a crunch period. This is also important to the team, as you can bring what you learn to the table to help in design of future corporate initiatives.

• Love for Mentoring

One of the most rewarding aspects of working on a development team can be the opportunity to help with the growth and learning of others on your team and teams you work with. There are many aspects to mentoring. Consistent application of the traits described in this blog series on leadership alone is a good way. From an early age on, people emulate the behavior of people they respect. Find other ways to personally connect with individuals on the team. Choose one or two on the team who are most engaged with you and work closely together. Often this will be more junior people on the team, often you will select people who show the most promise, but neither of these is a requirement. Remember that anyone with more experience in any one area can act as a mentor for anyone else in that area. You could easily end up helping someone older or with more overall experience, but less in one or more specific areas. Lastly, there are always opportunities to find others to help mentor you. Nobody is ever done learning and there are always people to learn from, whether on your team or externally. Try to spend as much time finding people whom you respect in an area you would like to grow in as you do in trying to find others who can learn from you.

• Balance and Perspective - Understanding of when and how much technical debt to accept

In constant conflict with your goal of delivering personal excellence will be the organization's need to deliver software in a timely fashion. A true leader will understand enough of the market dynamics to know when it is appropriate to "rush a solution" and when the constraints are artificial.
Organizations tend to constantly apply time pressure. You should understand when there is a true market force for a rush and when it is self-imposed by the organization. A clear-cut example of a true market need is delivery of software to a hardware product timeline owned by another organization or company. If there is a true market need, often you will be forced to create some "technical debt" and deliver software that is not as well written as you would like. In these situations, a leader will still strive for quality and test coverage in the solution, and sacrifice architectural and/or design integrity. In other situations, the time pressure may be internally-driven. You should act as the code-base custodian. Make sure that management understands the trade-offs of technical debt for this decision and weighs its long-term supportabilty costs. Importantly, a code-base custodian looks for appropriate times when external pressures are not as high to lobby for an investment in paying down technical debt from previous "rush jobs".

Key traits of a software leader - Personal Excellence, Integrity, Innovation, and Technology Evangelism

•  Personal Excellence

Software is continuously evolving and so must software professionals evolve. Find ways to continue to grow. There are numerous ways, including podcasts, books, blogs, meetups, technical conferences, code camps, and innovation challenges. Find a couple that fit your interest and style. Use resources like github and stack overflow to learn and experience other approaches to writing code and solving problems.

Leaders develop well-thought-out, well-architected, and well-tested software. Make everything you write something that you would feel comfortable publishing to open-source. and others on the team will emulate your style, template from your code, and learn from you.

• Integrity

Integrity is an important quality often overlooked in soft-skills discussions. Yet, integrity is arguably one of the most important aspects of leadership in any field. Never do anything unprofessional at work our outside of work with any of your colleagues. Treat all members of the team with respect and consideration...always. If you disagree with someone you should always make your voice heard, but your voice should be the voice of logic and reason, not the voice of emotion and ego. You don't need to be loved by everyone on the team but it helps to be well-liked, and you need to be highly respected.

Apply this integrity to your work ethic. You should provide aggressive schedule estimates that you can achieve with high confidence. You shouldn't have hidden agendas. Make choices and recommendations that are best for the team and organization. If this doesn't align with your goals, choose another team or organization that does.

•  Innovation

Innovation is perhaps one of the most difficult traits to learn. For an existing problem, it is easy to get caught up in what is familiar and to fit problems to existing toolsets and skills that you have always developed. Try to push beyond what is comfortable and known. Allocate some time before jumping into any task to thinking about what might be the best solution for your problem. Do this before providing schedule estimates.

Innovation also extends well beyond solving existing problems. Look around at the expertise of your group and your company. Periodically step back and think about other things that your group or company could invest in that would be a good fit for the company. Discuss some of these ideas with your manager or product manager and track them somewhere appropriate at your company, such as a wiki. If you strongly believe in your idea, don't be afraid to champion it or even spend some of your own time to develop a prototype.

•  Technology Evangelism

A strong leader will keep abreast of software technologies and advocate adoption within the team where appropriate. This should not become an exercise in constantly adopting the newest "hot" technology. Look for technologies which will improve the development cycle of your team. Would TDD improve the readability and maintainability of your team's code? Do you  have a complex system with frequent threading issues? Would a functional programming language improve the quality of your system? What about your revision control, bug tracking, and/or life-cycle tracking systems? Could your team benefit from improvements to these areas? Consider cost. Some of the best solutions are free. Remember that everything your team does is to improve the bottom line of the company. Often this is indirect and hard to quantify. Examples are investments in test, design, and architecture, which rarely benefit time-to-market, but often have a solid payoff in maintenance costs and ability to evolve the software over the lifetime of the product. Where possible, frame your recommendations in terms of cost savings.

Key traits of a software leader - Design, Quality, and Communication Tips

• Devotion and Knack for Design and Architecture

I am a strong advocate of refactoring throughout the lifecycle of a product. However, the cost of "fixing" a poor design can be very high and increases over time. It is always a good idea to do some level of design and architecture up-front, before any coding begins. There is a great deal of literature available on the benefits of good design that essentially boil down to software that is of higher quality, is easier to maintain, and has the flexibility to change over time. I believe it 's a mistake to think that Agile development precludes design and architecture. It is still important to include design and architecture in your agile processes. This may be a topic for a future post.

One example of something that must always be carefully considered is public interfaces. If your software project has a public-facing API, changing this API after release has a major impact on your customers. Of course you can version your APIs, but you will still be left supporting the old APIs for a long time, if not indefinitely. Poorly designed APIs can, thus, severely limit the growth and architectural flexibility of your code.

Bear in mind that software often lives in the field much longer than originally anticipated. I like to multiply the expected lifetime of the software by at least two when considering how much attention should  be given to design. Often times the end goal of a project is a prototype. Obviously for prototype work design is less important. Resist the surprisingly strong pull to allow a prototype to morph into a customer-facing product. If you work in an organization where prototypes frequently turn into product due to forces outside your control, you may need to start putting more design into your prototypes.

Lastly, while it is always more costly to do too little up-front design, there is also a cost of devoting too much time on design. This is a balance discussed in a future topic.

• Attention to Quality

This item speaks for itself. Apply modern coding techniques, and always remember the KISS principle. The highest quality approaches over the long-term are almost always the simplest approaches. Be an advocate for quality and simplicity within your team. As Hal Ableson and Gerald Sussman famously wrote, "Programs must be written for people to read, and only incidentally for machines to execute."

If you can't pick up something you wrote 3 months ago and understand what you wrote, chances are that code is also buggy. Two simple rules for code support that I find have the biggest impact are:

1. Name every method and class precisely
2. Decompose your methods as much as possible

These two techniques alone will make your code easier to read and understand.

Ensure quality through integration and unit tests. Measure test coverage and monitor tests frequently, if not with every checkin. Consider adopting a Test-Driven-Development(TDD) approach.

• Relentless and Effective Communicator

I find that face-to-face communications are always the most effective, but are not always practical, particularly with geologically diverse teams. Don't be afraid to hold meetings to discuss important design decisions, but don't waste people's times with frivolous ones. Pay attention to your attendee list. Invite all the stakeholders, but resist the urge to expand beyond that. If the list with stakeholders exceeds 5 or 6, consider if a smaller number of stakeholders would work better, and then later follow-up with the other stakeholders on the proceedings.

Where face-to-face is not possible, use video or phone. Next on my list would be instant messaging. Email tends to be one of the least effective means of communication. A good tactic to deal with an email chain greater than 5 or 10 replies is to setup a meeting to resolve the issues.

Being a leader does not mean directing everyone on the team on every aspect of design and coding. Recognize that there are many strong developers on your team and that everyone's view is valuable. The best leaders have well-respected voices in the discourse, providing strong influence on the group, not dictating to the group. Often others on the team have better or more practical solutions. A good leader will actively recruit other opinions on important matters and be quick to support good ones that are superior or equal to his/her own preconceived notions of solving the problem at hand. Many of us in software design have strong egos. I actually think this is important, because with ego comes confidence and a higher participation level. Be careful not to trample on others' egos and be careful to keep your own in check.

This extends even more importantly to cross-group communications. Always get input and buy-in from members of all affected teams on any decision which impacts or involves them. The best solutions tend to be the ones developed with a small group of people with differing backgrounds. There is no easier way to hit late roadblocks than to make a decision without including a major stakeholder.

Advance from team member to leader on your software development team - 10 Key Traits

How do you become a leader in a software organization?

In my experience, the qualities of a true leader don't vary much between work environments. This holds true from small startups with less than 10 employees to large companies with 100,000+ employees. I have worked in both extremes and had the pleasure of learning from many of these leaders.

I'd like to share my observation of ten of these qualities. While not every leader can be strong in all categories, I have seen all to be strong in a large subset. 

So, my top-ten list is...

• Devotion and Knack for Design and Architecture
• Attention to Quality
• Relentless and Effective Communicator
• Personal Excellence - constantly learning and growing
• Personal Integrity
• Innovative Thinker - bring unique perspectives to problems
• Technology Advocate
• Dedication
• Love for Mentoring
• Balance and Perspective - Understanding of when and how much technical debt to tradeoff vs market pressures

I have found that people who exhibit these attributes are recognized by members of their team and become de-facto "group leaders", increasing influence within and outside the organization. Other members of the team  look to these leaders for input, thoughts, and ideas.

In upcoming posts I will delve more deeply into these characteristics.