sglavoie.com – Tag: Skimming Notes

Reading notes: A Philosophy of Software Design, 2nd Edition

Sébastien Lavoie — Sun, 30 Mar 2025 11:18:00 -0600

Nature of complexity

“Change amplification: The first symptom of complexity is that a seemingly simple change requires code modifications in many different places.” (e.g., refactoring a project to inject dependencies in tests, because the system was hard to test in the first place)
“Cognitive load: The second symptom of complexity is cognitive load, which refers to how much a developer needs to know in order to complete a task.” (e.g., close a file where it is opened whenever possible, instead of leaking context and pushing the responsibility onto the caller)
“Unknown unknowns: The third symptom of complexity is that it is not obvious which pieces of code must be modified to complete a task, or what information a developer must have to carry out the task successfully.”
“One of the most important goals of good design is for a system to be obvious.”

Strategic vs. tactical programming

“it’s worth taking a little extra time to find a simple design for each new class; rather than implementing the first idea that comes to mind, try a couple of alternative designs and pick the cleanest one. Try to imagine a few ways in which the system might need to be changed in the future and make sure that will be easy with your design”

Modules should be deep

Modular design
- “In general, if a developer needs to know a particular piece of information in order to use a module, then that information is part of the module’s interface.”
Abstractions
- “The key to designing abstractions is to understand what is important, and to look for designs that minimize the amount of information that is important”
A deep module is one in which the interface is small and the implementation is large.
- “Deep modules such as Unix I/O and garbage collectors provide powerful abstractions because they are easy to use, yet they hide significant implementation complexity.”
Shallow modules
- “A shallow module is one whose interface is complicated relative to the functionality it provides.”

Information hiding

“If a piece of information is hidden, there are no dependencies on that information outside the module containing the information, so a design change related to that information will affect only the one module”
Information leakage
- “If the affected classes are relatively small and closely tied to the leaked information, it may make sense to merge them into a single class.”
- “When designing modules, focus on the knowledge that’s needed to perform each task, not the order in which tasks occur.”
- “it’s important to avoid exposing internal data structures as much as possible”
“The best features are the ones you get without even knowing they exist.”
“As a software designer, your goal should be to minimize the amount of information needed outside a module; for example, if a module can automatically adjust its configuration, that is better than exposing configuration parameters”
“think about the different pieces of knowledge that are needed to carry out the tasks of your application, and design each module to encapsulate one or a few of those pieces of knowledge. This will produce a clean and simple design with deep modules.”

General-Purpose Modules are Deeper

“specialization leads to complexity”
Generality leads to better information hiding
“Eliminate special cases in code”

Different Layer, Different Abstraction

“A pass-through method is one that does nothing except pass its arguments to another method, usually with the same API as the pass-through method. This typically indicates that there is not a clean division of responsibility between the classes.”

Pull Complexity Downwards

Figure out complexity in the library instead of pushing it up to the user.
"[…] you should avoid configuration parameters as much as possible. Before exporting a configuration parameter, ask yourself: will users (or higher-level modules) be able to determine a better value than we can determine here?"

Better Together Or Better Apart?

“Bring together if information is shared”
“Bring together if it will simplify the interface”
“Bring together to eliminate duplication”
- Use goto to exit nested code blocks early as long as it’s not too complex.
On Splitting long methods: “If you make a split of this form and then find yourself flipping back and forth between the parent and child to understand how they work together, that is a red flag (“Conjoined Methods”) indicating that the split was probably a bad idea.”
“If the caller has to invoke each of the separate methods, passing state back and forth between them, then splitting is not a good idea.”

Define Errors Out Of Existence

“The key overall lesson from this chapter is to reduce the number of places where exceptions must be handled; in some cases the semantics of operations can be modified so that the normal behavior handles all situations and there is no exceptional condition to report […].”
“The best way to eliminate exception handling complexity is to define your APIs so that there are no exceptions to handle: define errors out of existence”
E.g., the Java substring method could remove its IndexOutOfBoundsException by automatically rounding the index to the nearest valid value.
"Exception masking doesn’t work in all situations, but it is a powerful tool in the situations where it works. It results in deeper classes, since it reduces the class’s interface (fewer exceptions for users to be aware of) and adds functionality in the form of the code that masks the exception. Exception masking is an example of pulling complexity downward."
“The idea behind exception aggregation is to handle many exceptions with a single piece of code; rather than writing distinct handlers for many individual exceptions, handle them all in one place with a single handler.” (e.g., a web server dispatching different types of requests and catching all sorts of errors).
“In most applications there will be certain errors that are not worth trying to handle. Typically, these errors are difficult or impossible to handle and don’t occur very often. The simplest thing to do in response to these errors is to print diagnostic information and then abort the application.” (e.g., out-of-memory errors).

Design it Twice

“The initial design experiments will probably result in a significantly better design, which will more than pay for the time spent designing it twice”
“The process of devising and comparing multiple approaches will teach you about the factors that make designs better or worse. Over time, this will make it easier for you to rule out bad designs and hone in on really great ones.”

Why Write Comments? The Four Excuses

Good code is self-documenting. Comments serve as precisions on abstractions.
I don’t have time to write comments. The benefits of having documentation quickly offset the cost of writing it.
Comments get out of date and become misleading. Keep the documentation close to where it needs to be, so you update it.
All the comments I have seen are worthless. Be the change you want to see.
Benefits of well-written comments
- “The overall idea behind comments is to capture information that was in the mind of the designer but couldn’t be represented in the code.”
- “One of the purposes of comments is to make it it unnecessary to read the code.”

Comments Should Describe Things that Aren’t Obvious from the Code

Don’t repeat the code
“Lower-level comments add precision”
- Focus on what the variable represents, not how it is modified (the latter is explained by the code).
“Higher-level comments enhance intuition”
- “What is the simplest thing you can say that explains everything in the code? What is the most important thing about this code?”
Interface documentation
“Implementation comments: what and why, not how”
- Describe at a more abstract level to help understand what is happening and why. The code tells the how.
Cross-module design decisions
- One possible solution is to use a designNotes.md file or similar to jot down all major decisions, and point to it from the code (e.g., See topic in designNotes).

Choosing names

Great names are “precise, unambiguous, and intuitive.”
Use names consistently: make it so that a concept is expressed in the same words regardless of the context in which it is found if it makes sense to do so.
Avoid extra words: no more Hungarian Notation with modern IDEs.

Use comments as part of the design process

Delayed comments are bad comments
- You won’t remember exactly what you had in mind and why you did things if you delay writing documentation: design decisions will get lost.
Write the comments first
- No backlog of unwritten comments.
- “it produces better comments.”
Comments are a design tool
- Writing them at the beginning “improves the system design.”
- “If a method or variable requires a long comment, it is a red flag that you don’t have a good abstraction.”
Early comments are fun comments
- “If you are programming strategically, where your main goal is a great design rather than just writing code that works, then writing comments should be fun, since that’s how you identify the best designs.”
- “I’m looking for the design that can be expressed completely and clearly in the fewest words. The simpler the comments, the better I feel about my design.”
Are early comments expensive?
- “Writing the comments first will mean that the abstractions will be more stable before you start writing code.”
- “In contrast, if you write the code first, the abstractions will probably evolve as you code, which will require more code revisions than the comments-first approach.”

Modifying existing code

“Whenever you modify any code, try to find a way to improve the system design at least a little bit in the process. If you’re not making the design better, you are probably making it worse.”
Maintaining comments: keep the comments near the code
- “The farther a comment is from its associated code, the less likely it is that it will be updated properly.”
- “In general, the farther a comment is from the code it describes, the more abstract it should be (this reduces the likelihood that the comment will be invalidated by code changes).”
Comments belong in the code, not the commit log
- “An example is a commit message describing a subtle problem that motivated a code change. If this isn’t documented in the code, then a developer might come along later and undo the change without realizing that they have re-created a bug.”
Maintaining comments: avoid duplication
Maintaining comments: check the diffs
- Review changes during the pre-commit phase to make sure documentation stays up-to-date with the code changes.
Higher-level comments are easier to maintain
- Most helpful comments do not repeat the code, they provide an abstraction that can’t be expressed easily by the code alone.

Consistency

Similar things are done in a similar way.
“Consistency creates cognitive leverage: once you have learned how something is done in one place, you can use that knowledge to immediately understand other places that use the same approach.”
Ensuring consistency
- Document most important conventions.
- “the value of consistency over inconsistency is almost always greater than the value of one approach over another.”

Code should be obvious

“the best way to determine the obviousness of code is through code reviews”
“If someone reading your code says it’s not obvious, then it’s not obvious, no matter how clear it may seem to you”
Things that make code more obvious
- Good names
- Consistency
- Judicious use of white space
- Blank lines
- Comments
Things that make code less obvious
- Event-driven programming
- Generic containers
- Different types for declaration and allocation

Software trends

Object-oriented programming and inheritance
- “In order for an interface to have many implementations, it must capture the essential features of all the underlying implementations while steering clear of the details that differ between the implementations; this notion is at the heart of abstraction.”
- “Class hierarchies that use implementation inheritance extensively tend to have high complexity.”
- “Before using implementation inheritance, consider whether an approach based on composition can provide the same benefits.”
Agile development
- May encourage a more tactical approach to programming. Balance is required.
Test-driven development
- “it focuses attention on getting specific features working, rather than finding the best design”
- “One place where it makes sense to write the tests first is when fixing bugs. Before fixing a bug, write a unit test that fails because of the bug.”
Design patterns
- “As with many ideas in software design, the notion that design patterns are good doesn’t necessarily mean that more design patterns are better.”
Getters and setters
- “It’s better to avoid getters and setters (or any exposure of implementation data) as much as possible.”

Designing for performance

How to think about performance
- “The key is to develop an awareness of which operations are fundamentally expensive.”
Measure before (and after) modifying
- “the goal is to identify a small number of very specific places where the system is currently spending a lot of time, and where you have ideas for improvement.”
- “If the changes didn’t make a measurable difference in performance, then back them out (unless they made the system simpler).”
Design around the critical path
- “When redesigning for performance, try to minimize the number of special cases you must check.”
“Complicated code tends to be slow because it does extraneous or redundant work.”

Decide what matters

Minimize what matters
How to emphasize things that matter
- “important things should appear in places where they are more likely to be seen, such as interface documentation, names, or parameters to heavily used methods.”
- “repetition: key ideas appear over and over again.”
- “The things that matter the most should be at the heart of the system, where they determine the structure of things around them.”
Mistakes
- “treat too many things as important. When this happens, unimportant things clutter up the design, adding complexity and increasing cognitive load.”
- “fail to recognize that something is important. This mistake leads to situations where important information is hidden, or important functionality is not available so developers must continually recreate it.”
Thinking more broadly
- “the best way to make a document easy to read is to identify a few key concepts at the beginning and structure the remainder of the document around them. When discussing the details of a system, it helps to tie them back to the overall concepts.”
- “Focusing on what is important is also a great life philosophy: identify a few things that matter most to you, and try to spend as much of your energy as possible on those things.”

Conclusion

“The reward for being a good designer is that you get to spend a larger fraction of your time in the design phase, which is fun. Poor designers spend most of their time chasing bugs in complicated and brittle code.”

Book summary: 100 Go Mistakes and How to Avoid Them

Sébastien Lavoie — Sat, 24 Aug 2024 14:55:00 -0600

Go: Simple to learn but hard to master

Go outline

Go does not have:
- Type inheritance;
- Macros;
- Partial functions;
- Support for lazy variable evaluation;
- Support for immutability;
- Operator overloading;
- Pattern matching;
- …
Go is:
- Stable;
- Expressive;
- Fast to compile;
- Safe (type-safety);
- Simple;
- Excellent for concurrent programming (goroutines, channels).

Simple doesn’t mean easy

Concurrency in Go may be simpler than in other languages, but it is still hard to get right, as per this study.

100 Go mistakes

Learning from mistakes is efficient.
“the best time for brain growth is when we’re facing mistakes” (source).
“we can remember not only the error but also the context surrounding the mistake” (source).
Categories of mistakes
- Bugs
  - Bugs are very costly (source).
  - Bugs can be very dangerous (e.g., in the case of a radiation therapy machine).
- Needless complexity
  - Don’t over-complicate things.
- Weaker readability
  - “the ratio of time spent reading versus writing is well over 10 to 1” (source).
  - Nested code.
  - Data type representation.
  - Not using named result parameters.
- Suboptimal or unidiomatic organization
  - Utility packages.
  - Init functions.
- Lack of API convenience
  - Overusing any types.
  - Wrong creational pattern.
  - Blindly applying OOP principles.
- Under-optimized code
  - Lack of fundamental knowledge.
  - Impacted performance.
  - Floating-point accuracy.
  - Poorly parallelized execution.
  - Not knowing how to reduce allocations.
  - Data alignment.
- Lack of productivity
  - Know how to write efficient tests.
  - Use the standard library.
  - Use profilers and linters.

Code and project organization

1 - Unintended variable shadowing

A variable name declared in a block can be re-declared in a nested block (variable shadowing).

 go
 
 
 
// client is shadowed
var client *http.Client // will always be nil
 if tracing {
 client, err := createClientWithTracing()
 if err != nil {
 return err
 }
 log.Println(client)
 } else {
 client, err := createDefaultClient()
 if err != nil {
 return err
 }
 log.Println(client)
 }

Solutions
- Use a temporary variable name in the inner scope.
- Use the = operator instead of := in the inner scope. Leads to a single assignment. Error handling can be handled in the outer scope (client, err = ...).
- Be caution with reusing err as well.

2 - Unnecessary nested code

Put the happy path first and return early on all edge cases and errors.
Instead of using else blocks, use early returns.
Instead of checking for an happy path, check for the error path and return where possible.

3 - Misusing init functions

Refresher on init functions
- An init function is a special function that is executed before the main function. It takes no arguments and returns no values.
- Variables are initialized, the init function is called, and then the main function is called.
- Imported packages are initialized before the main function is called and before the init function in the main package is called because the main package depends on the imported packages.
- Packages can define multiple init functions, in which case they are executed in alphabetical order based on the source code file name. Hence, we should not depend on the order of execution of init functions as file names can change.
- The same source file can define multiple init functions, in which case they are executed in the order they are defined in the file.
- The underscore operator (_) can be used to import a package without using it. This is useful to execute the init function of a package without using the package itself.
- init functions cannot be invoked explicitly.
When to use init functions
- They should not be used to initialize variables, as error handling is limited (they don’t return errors, one must panic).
- When testing a package, the init function is executed before the tests are run.
- If side effects are achieved via variables, these become global variables, which pollute the global namespace. Any function in the package can access these variables, which can lead to unexpected behavior. Unit tests depending on such functions will be harder to write.
- The responsibility of error handling should be left up to the caller.
- They can be helpful to define static configurations, as is done in the init function of the Go blog.

4 - Overusing getters and setters

Go has no support for automatic getters and setters, and it is idiomatic to use direct access to fields.
When the use case justifies it, a getter should be named FieldName (not GetFieldName) and a setter should be named SetFieldName.

5 - Interface pollution

Go’s interfaces are satisfied implicitly (structural typing).
“The bigger the interface, the weaker the abstraction.” — Rob Pike
When to use interfaces
- Common behavior
  - The behavior is factored out into an interface when it is common to multiple types (e.g., sort.Interface).
- Decoupling
  - The implementation is decoupled from the interface, which allows for easier testing and swapping of implementations.
- Restricting behavior
  - The interface restricts the behavior of the type, which can be useful for enforcing constraints. For example, the io.Reader interface restricts the behavior of the type to reading. It is possible to enforce constraints such as with a getter interface that only has a Get method and prevents setting values.
Interface pollution
- Abstractions should be discovered, not invented. Only create interfaces when they are needed.
- Adding levels of indirection makes the code harder to understand.

6 - Interface on the producer side

Producer side refers to an interface defined in the same package as the type that satisfies it.
Consumer side refers to an interface defined in a package that uses it and which is independent of the package that implements the interface.
The client should determine the level of abstraction it needs.
This relates to the Interface segregation principle of SOLID.

7 - Returning interfaces

In many cases, this is a bad practice in Go.
“Be conservative in what you do, be liberal in what you accept from others.” — Transmission Control Protocol
- Return structs instead of interfaces.
- Accept interfaces if possible.
Interfaces might be returned (e.g., io.Reader), but this should be done with caution. It’s usually done when the interface itself is implemented on the producer side (e.g., standard library) and the consumer side (e.g., user code) is expected to use it.

8 - `any` says nothing

The interface{} type is a placeholder for any type. any is an alias for interface{} since Go 1.18.
It requires a type assertion to be used.
Using any does not convey any meaningful information about the type.
It will lead to issues compilation-wise.

9 - Being confused about when to use generics

A “type parameter” is a placeholder for a type that is specified when the function is called.
A constraint is a condition that the type parameter must satisfy (e.g., implement an interface like comparable).

 go
 
 
 
type intStringConstraint interface {
 ~int | ~string
 // ^--- union operator
}

The union operator (|) is used to specify that the type parameter must be either int or string.
The tilde (~) is used to restrict all the underlying types of the type parameter to the specified types.
Common uses and misuses
- Generics are useful to factor out common behavior (e.g, the sort interface).
  - Data structures (e.g., binary tree, linked list, heap).
  - Functions working with slice, map, channels of any type.
- Generics are not useful when the behavior is not common.
  - If the method of the type argument is called, then the generic should be removed in the function received and a type should be specified in the function parameter.
- Generics do not shine when they make the code more complex. Unnecessary abstraction should be avoided.

10 - Not being aware of the possible problems with type embedding

A type embedding is a way to reuse the fields and methods of a type in another type. The fields from that other type get “promoted” to the new type (i.e., they can be accessed directly from the new type, as well as through the embedded type).
Incorrect usage
- An example of a wrong use of type embedding is a sync.Mutex embedded in a struct. It will be available outside the struct and can be misused.
- It shouldn’t be used only as syntactic sugar to access the methods of the embedded type. Use a field instead.
- It shouldn’t promote fields that are not intended to be used outside the struct (e.g., sync.Mutex).
Correct usage
- A custom logger that wants to write and close may embed a io.WriteCloser interface so that both methods are available. They will be promoted to the new type and there won’t be a need to implement these methods simply to forward a call to the embedded type.
- Preventing type embeddings in public structs is probably the safest route.
Embedding vs. OOP subclassing
- Embedding is not subclassing. It is a way to reuse fields and methods of a type in another type. It is a form of composition.

11 - Not using the functional options pattern

Config struct
- This is limited because to bring flexibility and distinguish between the zero value and the default value, one must use pointers.
Builder pattern
- To keep chaining method calls, one should not return an error, which means validation is delayed.
- To have proper error handling along the way, it makes the configuration more complex than it needs to be.
Functional options pattern
- This pattern makes use of closures to set the configuration options, where the closure is an anonymous function being returned.
- The closure is called with the configuration struct as an argument, and it sets the configuration options.
- This pattern allows for validation of the configuration options and for the configuration to be set in a single call.
- It is a flexible pattern that allows for the configuration to be set in any order and for the configuration to be validated at the time of setting.
- One does not need to provide a default value for the configuration struct since the function accepts variadic arguments, which may be empty.

12 - Project misorganization

Project structure
- The project-layout is a great place to start for a complex application. Despite not being an official standard, it is a great and logically sound starting point.
Package organization
- There is no concept of subpackages in Go. Sub-directories are useful to have high cohesion.
- Hexagonal architecture group per technical layer.
- An alternative is to group by context (e.g., customer context, contract context).
- Avoid both nano and huge packages.
  - Nano packages probably mean there’s lack of understanding about the connections between the different files.
  - Huge packages dilute the meaning of the package.
  - Name a package after what they provide, not what they contain.
  - “Package names should be short, concise, expressive and, by convention, a single lowercase word”.
- Minimize what should be exported as much as possible to reduce coupling.

13 - Creating utility packages

utils, common, base… bad practice.
util is meaningless.
While nano packages should be avoided, if a few functions are highly cohesive and don’t belong anywhere else, it’s a good idea to keep them separate with good naming.
Make APIs expressive.

14 - Ignoring package name collisions

Naming a variable the same as a package name will shadow the package name.
Either rename the variable (redisClient := redis.NewClient()) or rename the import with an alias (import redisapi "mylib/redis").
The same is true for built-in functions like copy: they shouldn’t be shadowed.

15 - Missing code documentation

Simplifies how clients consume an API.
Helps in maintaining a project.
In Go, every exported element should be documented, starting with the name of the element being documented (e.g., Customer is ...). It should be a complete sentence ending with a period.
An element can be marked as deprecated with a comment starting with Deprecated: after the description of the element.
The convention to document a package is to have a comment at the top of the file with the package name and a description of the package, starting with // Package packageName .... The first line should be concise, then a blank line, then a more detailed description.
Comments at the top of the file will be omitted from the documentation if they are followed by a blank line and more comments.

16 - Not using linters

Automatic tool to analyze code and catch errors.
They help prevent mistakes automatically and consistently.

Data types

17 - Creating confusion with octal literals

Using 0o instead of 0 for octal literals is clearer.
Octal is still useful to express file permissions (e.g., 0644).
Go supports using underscores in numeric literals to improve readability (e.g., 1_000_000).

18 - Neglecting integer overflows

It won’t panic at runtime. They are silent bugs.
It will wrap around.
Use the math package to check for overflows.
Detecting integer overflow when incrementing
- Use math.MaxInt, math.MinInt, math.MaxUint, math.MinUint.
Detecting integer overflows during addition
- Use math.MaxInt - a < b or math.MaxUint - a < b.
Detecting integer overflows during multiplication
- Use math.MaxInt/a < b or math.MaxUint/a < b.

19 - Not understanding floating points

A mantissa is the significant part of a floating-point number.
The exponent is the power of 10 by which the mantissa is multiplied.
For float32, the mantissa is 23 bits and the exponent is 8 bits.
For float64, the mantissa is 52 bits and the exponent is 11 bits, the remaining bit is the sign bit.
It is best to avoid comparing by equality with floating-point numbers and instead use a tolerance (delta).
Use math.IsInf and math.IsNaN to check for infinity and NaN.
Group floating-point numbers together by their magnitude to reduce precision loss when adding and subtracting.
With calculations involving both addition/subtraction and multiplication/division, it is best to perform the multiplication/division first to reduce precision loss.

20 - Not understanding slice length and capacity

Slice data is stored contiguously in an array data structure.
A slice holds a pointer to the array, the length of the slice, and the capacity of the slice.
Trying to access an index greater than the length of the slice will result in a runtime panic, even though the capacity is greater.
Slicing a slice will create a new slice with the same underlying array, but with a different length and capacity.
The append function will create a new slice with a new underlying array if the capacity is exceeded.

21 - Inefficient slice initialization

Use make to initialize a slice with a specific length and capacity.
Set a length of 0 (2nd argument to make) and a capacity of n (3rd argument to make), which will reuse the same backing array until the capacity is exceeded.
One can also set only the length and use direct assignment instead of append (bars[i] = fooToBar(foo)). This is about 400% faster than using an empty slice.
If the length of the slice is already known, it is better to take advantage of it and set the length of the slice when initializing it.

22 - Being confused about nil vs. empty slices

A slice is empty if its length is equal to 0.
A slice is nil if it equals nil.
A nil slice is an empty slice.
A nil slice can be passed in a single line: s := append([]int(nil), 42).
A nil slice is marshaled as a null element, whereas a non-nil, empty slice is marshaled as an empty array.

23 - Not properly checking if a slice is empty

"[…] when designing interfaces, we should avoid distinguishing nil and empty slices, which leads to subtle programming errors".
“This principle is the same with maps. To check if a map is empty, check its length, not whether it’s nil”.
Do len(slice) != 0 instead of slice != nil.

24 - Not making slice copies correctly

The destination slice must have the same length as the source slice (or greater). Else, the copy will match the length of the destination slice and may be missing elements.
copy is the more idiomatic way to copy slices (vs. dst := append([]int(nil), src...)).

25 - Unexpected side effects using slice `append`

If the resulting slice as a length smaller than the capacity, the slice will be modified in place and will have unintended side effects.
To avoid side effects outside a given range of a slice, full slice expression can be used (e.g., s := []int{1, 2, 3}, then s[:2:2] will create a new slice with a length of 2 and a capacity of 2).

26 - Slices and memory leaks

Leaking capacity
- Slicing a huge slice to a smaller slice will keep the capacity of the original slice. This makes matters worst if the result of a slice is kept for many iterations in a loop.
- Instead, use copy to create a new slice with the necessary length.
Leakage with full slice expressions
- The issue is the same as with slicing.
- The solution is the same as with slicing.
Slice and pointers
- A slice holds a pointer to an array, so if the slice is kept, the array will be kept in memory.
- To avoid this, use copy to create a new slice with the necessary length.
- If however we want to keep the original capacity, we can set all the unneeded elements to nil explicitly. For instance, all elements but the first 2:

 go
 
for i := 2; i < len(foos); i++ {
 foos[i].v = nil
 }

27 - Inefficient map initialization

Maps are unordered collections of key-value pairs, where all keys are unique. They use hash tables under the hood.
Insertion can be an O(n) operation if the map is resized.
make can be used to initialize a map with a specific size (second argument to make, without capacity). This avoids having to grow the map and re-balancing all the buckets of 8 elements each.

28 - Maps and memory leaks

The number of buckets in a map cannot shrink. Removing elements from a map will not reduce the number of buckets.
Even the overflown buckets will not be reduced.
One possible solution: periodically copy the map to a new map.
- Downside: the amount of memory used will be doubled for a short period of time.
Another solution: update the value type of the map to store an array pointer.
- The peak memory consumed will be far less.
- The garbage collection will be more efficient.

29 - Comparing values incorrectly

== does not work for slices, maps, and functions.
reflect.DeepEqual could be used, but it’s about 100x slower than ==, and so might be better used for testing only.
If strict comparison is needed on structs and good performance is required, consider using a custom comparison function.

Control structures

30 - Ignoring the fact that elements are copied in range loops

In Go, every assignment leads to a copy of the value.
In a range loop, the value is copied, not the reference.
To update a slice with a range loop, one must use the index to update the slice, i.e., for i := range s { s[i] = ... }.
A classic for loop can be used to update the slice in place, i.e., for i := 0; i < len(s); i++ { s[i] = ... }.
Using a pointer to a slice will allow for the slice to be updated in place, but iterating over such a slice is less efficient for the CPU.

31 - Ignoring how arguments are evaluated in range loops

Looping over an array will copy the value of the array element.
Using a pointer to the array will allow for the array to be updated in place and will avoid copying values, which may be more efficient, especially on large arrays.
The range loop evaluates the expression to its right only once, so if the expression is a slice, the slice will not be updated in the loop.
To access updated elements of a slice during iteration with a range loop, one must use the index to access the element in the slice, not the value of the element, since the expression is copied to a temporary variable.

32 - Ignoring the impact of using pointer elements in range loops

Using a pointer to a slice when looping with a range loop can lead to storing the address of the temporary variable in the slice for all elements in the slice.
To avoid this issue, one can create a local variable for each element to be iterated over in the slice and then use the local variable in the loop to perform assignments.
Another solution is to use a range loop with the index, create a temporary variable by dereferencing the pointer, and then assign the temporary variable to the slice.

33 - Making wrong assumptions during map iterations

Maps don’t sort keys.
Insertion order is not preserved.
Iteration order is unspecified. This is a conscious language design choice so that developers don’t rely on the order of the keys.
One should work on a copy of a map if new elements are to be added or removed during iteration.

34 - Ignoring how the break statement works

break works on the innermost loop, whether that be a for, switch, or select.
To break outside of a loop, one can use a label.
A label can be an idiomatic way to break out of nested loops, as is done in the net/http package with a readlines: label.

35 - Using `defer` inside a loop

defer is a statement that schedules a function to be called when the surrounding function returns. That is, it won’t run on each iteration of the loop until it returns, if ever.
To avoid resource leaks like this, each iteration of the loop can use a separate function that is run and defers the cleanup.
Another solution is to use a closure, but this can be less readable.
A function call will decrease performance slightly, so if performance is a concern, the defer call should be handled manually before the loop.

Strings

36 - Not understanding the concept of a rune

A charset is a set of characters.
An encoding describes how to translate a character set into a sequence of bytes.
In Go, a string is a reference to an immutable slice of bytes.
A rune is a Unicode code point.
Source code in Go is encoded in UTF-8: string literals will then also be encoded in UTF-8.
UTF-8 is a variable-length encoding, where each code point is represented by one to four bytes. For instance, the string hello is represented by five bytes, but the string containing the character 汉 is represented by three bytes, which we can check with len("汉").
A rune is an alias for int32 (4 bytes = 32 bits).
The repository golang.org/x contains extensions to work with UTF-16 and UTF-32.

37 - Inaccurate string iteration

With the unicode/utf8 package, one can use utf8.RuneCountInString(s) to count the number of runes in a string.
Accessing specific runes can be done without converting the string if characters are in the ASCII range.
It is most useful to convert a string to a slice of runes when a specific rune is required by index.
Iterating over the runes with a for range loop returns the starting index of the rune and the rune itself.
Iterating with a for range using only the index returns the starting index of the rune and the byte value at that position.

38 - Misusing trim functions

Use TrimRight to remove trailing characters. Go backwards until a rune is not in the set of characters to be trimmed.
Use TrimSuffix to remove a suffix (only one substring occurrence).
Use TrimLeft to remove leading characters. Go forwards until a rune is not in the set of characters to be trimmed.
Use TrimPrefix to remove a prefix (only one substring occurrence).

39 - Under-optimized string concatenation

+= is inefficient for string concatenation because it creates a new string each time.
Use strings.Builder for efficient string concatenation.
Call Grow on the builder to allocate memory for the string.

40 - Useless string conversions

Working with []byte is more efficient than working with strings due to many interfaces in the standard library taking []byte as input, which avoids doing a conversion.
Most methods in the strings package have a []byte counterpart in the bytes package.

41 - Substrings and memory leaks

A substring shares the same backing array as the original string.
If the substring is kept, the original string will be kept in memory.
To avoid this, either use string([]byte(theString[:n])) to slice the needed part of the string or strings.Clone to create a new string.

Functions and methods

42 - Not knowing which type of receiver to use

Go does not pass values by reference.
A value receiver makes a copy of the value: changes are local to the method.
A pointer receiver passes the memory address of the value: changes are global to the value. Strictly speaking, this is still a copy, but the copy is made for a pointer.
Use a pointer receiver when:
- The method needs to mutate the receiver.
- The method receiver contains a field that cannot be copied.
- The receiver is a large struct.
Use a value receiver when:
- Enforcing immutability.
- The receiver is a map, func or chan.
- A slice that doesn’t have to be mutated.
- The receiver is a small struct or array with no mutable fields, like time.Time.
- The receiver is a basic type, like int, float64, or string.
Mixing value and pointer receivers can lead to confusion and bugs: this should generally be avoided (although the standard library shows some exceptions, like time.Time).
Defaulting to a value receiver is a good practice, unless there is a good reason to use a pointer receiver.

43 - Never using named result parameters

Only use named result parameters when they make the code more readable (e.g., returning two int can be ambiguous).
Use naked returns only when the function is short and the return values are clear.
For an interface definition, named result parameters are useful to document the return values.

44 - Unintended side effects with named result parameters

Named result parameters are initialized to their zero value.
Using named result parameters doesn’t mean one has to use naked returns: clarity is more important than brevity.

45 - Returning a nil receiver

A nil receiver is a pointer to a struct that has not been initialized.
Instead of returning a pointer to a struct (which will return a non-nil pointer to a nil struct), return nil directly.

46 - Using a filename as a function input

It is more idiomatic and flexible to use an io.Reader or io.Writer as an input to a function that reads or writes to a file: this abstracts the data source.
This allows for the function to be used with any type that implements the io.Reader or io.Writer interface, not just a file.
It makes testing easier, as one can pass in a bytes buffer instead of a file, for instance.

47 - Ignoring how defer arguments and receivers are evaluated

The arguments to a defer statement are evaluated when the defer statement is executed, not when the deferred function is executed.
- That means that if a variable is initialized with a zero value and passed to defer as is, the deferred function will receive the zero value.
- To avoid this pitfall, one can use a pointer to the variable, which will be evaluated when the deferred function is executed. The memory address will not change, but the value at that address will.
- Another solution is to use a closure to capture the value of the variable at the time the defer statement is executed.
  - This works because the values of the variables are evaluated when the closure runs, not when the closure is created.
In the case of a method receiver, data in the receiver is evaluated when the defer statement is executed, leading to the same situation as with arguments.
- A pointer receiver can be used to avoid this issue, as the updated value of the pointer will be visible to the deferred function when it is executed.

Error management

48 - Panicking

It is used to signal genuinely unexpected errors, such as programming errors (e.g., net/http checking for a status code outside the valid range).
Another use case for panic is when the application requires a dependency but fails to initialize it (e.g., a regular expression).

49 - Ignoring when to wrap an error

%w is used to wrap an error conveniently.
The %v directive will print the transformed error message, but %w will print the error message and the wrapped error message.
If a caller relies on unwrapping an error provided with %w, it leads to coupling: if the underlying implementation that is wrapped changes, the caller will break.
To mark an error, one should use a custom error type.

50 - Checking an error type inaccurately

errors.As is used to check if an error is of a specific type anywhere in the error chain.
errors.As should be used whenever error wrapping is used.

51 - Checking an error value inaccurately

A sentinel error is a predefined error that can be compared to an error value. It is about an “expected” error (e.g., sql.ErrNoRows and io.EOF).
Unexpected errors should be implemented as error types, implementing the error interface.
errors.Is is used to check if an error is equal to a sentinel error (not ==! if the error is wrapped, it will not be equal to the sentinel error).

52 - Handling an error twice

Logging the same error twice can lead to confusion.
Logging an error is handling an error! One should not both log and return an error.
Error wrapping is useful to provide context to an error when propagating it up the call stack.

53 - Not handling an error

The only way to explicitly ignore an error is to assign it to the blank identifier _. Adding a comment about why the error is ignored is a good practice.
Do not simply ignore the assignment.

54 - Not handling defer errors

Errors should be handled or explicitly ignored in defer functions.
To return an error to the caller via the defer function, one can use a named return parameter, assigning the error to it.
Instead of returning multiple errors from a defer function, it may be a good idea to log one error and return the other, depending on the context.

Concurrency: Foundations

55 - Mixing up concurrency and parallelism

Concurrency is about structuring a program, while parallelism is about running multiple tasks at the same time.
Concurrency enables parallelism.

56 - Thinking concurrency is always faster

A thread is the smallest unit of execution that can be scheduled by an operating system.
A goroutine is a lightweight thread managed by the Go runtime: it is context-switched on and off by an OS thread.
A goroutine has a memory footprint of 2 KB, while a thread has a memory footprint of ~2 MB.
There is a point of diminishing returns when creating too many goroutines: one must take into account the overhead of context switching and determine the size of the workload that will be processed by each goroutine.
It is often best to start with a sequential solution and use profiling tools to determine where to add concurrency.

57 - Being puzzled about when to use channels or mutexes

Synchronization between parallel goroutines is achieved with mutexes (mutex stands for mutual exclusion).
Channels are used to communicate between goroutines. If there are 3 goroutines and a third one needs results from the first two, a channel is the way to go.
Mutexes ensure exclusive access to a resource (e.g., a map).
Coordination or ownership of a resource is achieved with channels.
In general, mutexes are needed for parallel goroutines, whereas channels are needed for concurrent ones.

58 - Not understanding race problems

A data race occurs when two goroutines access the same variable concurrently and at least one of the accesses is a write.
sync/atomic provides atomic operations to avoid data races. It works only on primitive types.
Channels can be used to perform atomic operations on non-primitive types, then send the result over the channel.
Mutexes can be used to protect a critical section of code.
The absence of a data race doesn’t mean the code will be deterministic: it is possible for goroutines to be scheduled in different orders, leading to a race condition.
A buffered channel can lead to a data race because a receive from an unbuffered channel happens before the send, but a send to a buffered channel happens before the receive.

59 - Not understanding the concurrency impacts of a workload type

A worker pool can be based on GOMAXPROCS (the number of OS threads) to efficiently distribute the workload when a workload is CPU-bound.
When a workload is I/O-bound, the number of goroutines can be increased to handle more I/O operations concurrently, depending on the external systems.

60 - Misunderstanding Go contexts

A context is a way to pass deadlines, cancellation signals, and other request-scoped values across API boundaries and between processes.
context.Background is the root context, which is never canceled, has no values, and has no deadline, used when there is no parent context.
Deadline
- A deadline is a point in time when the context should be canceled, specified as either a time.Time or a time.Duration (time from now).
- Deferring the cancellation of a context can prevent a goroutine from being blocked indefinitely or leaking resources.
Cancellation signals
- context.WithCancel returns a copy of the parent context with a new Done channel.
Context values
- context.WithValue returns a copy of the parent context with a key-value pair.
A function that users wait for should take a context as an argument.
Prefer context.TODO() to context.Background() when a context is needed but there is no parent context, as it better represents the intent.

Concurrency: Practice

61 - Propagating an inappropriate context

Propagating a context is about passing a context from one goroutine to another.
A context should be passed to a goroutine carefully: if the context requires some values to be passed but the parent goroutine gets its context cancelled, the child goroutine will also be cancelled. This may be undesirable.
- To avoid this, create a custom struct implementing a context that never expires and that does not carry a cancellation signal, so the values can be passed without the risk of being cancelled.

62 - Starting a goroutine without knowing when to stop it

A goroutine might loop over the values received from a channel, but if the channel is never closed, this will be a resource leak.
Passing a context to a goroutine (e.g., to expect a file to be closed) is bad design. Instead, the parent goroutine should be responsible for closing the file by calling a defer function right after the resource used by another routine is created.

63 - Not being careful with goroutines and loop variables

Spinning up a goroutine inside a for loop and relying on the loop variable can lead to unexpected behavior.
Either assign the value of the loop variable to a new variable inside the loop or pass the loop variable as an argument to the goroutine. In the first case, the goroutine could be set up as a closure. In the second case, it would be a regular function call on an anonymous function.

64 - Expecting deterministic behavior using select and channels

The select statement is used to wait on multiple channels. If multiple channels are ready, one will be chosen semi-randomly.
Using an unbuffered channel with a select statement will block until a value is sent on the channel.
Using a single channel and then checking the message type is an option.
Using a default case in a select statement will allow the program to continue if a channel no longer receives values.
- This approach works well for multiple channels: when a select case is chosen, add an inner select statement to handle the channel. For instance, after a disconnection message, handle all remaining messages sent the another channel by looping over the channel inside that select case.

65 - Not using notification channels

A notification channel is a channel that is used to signal the completion of a task.
If no data is needed to be sent, a struct{} can be sent over the channel. This is clearer than sending a bool or int. An empty struct occupies no memory.

66 - Not using nil channels

Sending a message to a nil channel will block forever.
Receiving from a closed channel is a non-blocking operation: the zero value of the channel type will be returned.
When looping over multiple channels and using select, if one of the channels is closed, this will lead to a busy-wait loop.
- E.g., say we have two channels in a function that are to be “merged” such that data returned from that function comes from the two channels. If one of the channels is closed, the function will still keep looping over the now invalid select case.
To avoid this, one can use a nil channel to signal a channel is closed. The select case will be skipped if the channel is nil, avoiding the busy-wait loop.

67 - Being puzzled about channel size

An unbuffered channel (synchronous channel) has a size of 0: the sender blocks until the receiver receives data from the channel.
With a buffered channel, the sender will block only when the buffer is full.
An unbuffered channel enables synchronization between goroutines.
An unbuffered channel can be used as a notification channel via a channel closure (close).
When using worker pools, a buffered channel can be used to limit the number of tasks that can be processed concurrently, matching the number of goroutines in the pool.
Rate-limiting can be achieved with a buffered channel, where the buffer size is the number of tasks that can be processed concurrently.
In a buffered channel, the lower the value, the bigger the impact on CPU. The higher the value, the bigger the impact on memory.
Buffered channels can lead to obscure deadlocks.
It’s usually best to start with a channel of size 1 and perform benchmarks to determine the optimal size.
Queues tend to be close to full or close to empty anyways (source).

68 - Forgetting about possible side effects with string formatting

Printing the values in a context will evaluate the values at the time of the call, which could lead to a data race.
Acquiring a lock only when it’s required is best. This reduces side effects and possible deadlocks. This ties nicely with the fact that a function can go over the “error cases” before doing anything else that might require more resources (e.g., check the age of a customer is valid before acquiring a lock).

69 - Creating data races with append

To prevent data races with append, one can make a copy of the slice in each goroutine before appending to it.
If the slice is initialized in such a way that the backing array is shared between goroutines, there will not be a data race since the original slice will remain unchanged.
Updating a map from one goroutine will lead to a data race if another goroutine tries to read from the map at the same time, given that the hashing algorithm for maps introduces some randomness.

70 - Using mutexes inaccurately with slices and maps

A mutex is used to protect a critical section of code from being accessed by multiple goroutines at the same time.
A mutex is used to protect a shared resource, like a slice or a map.
If an operation is lightweight, it is better to use a mutex to protect the resource (protecting the whole function in the process).
If an operation is heavy, it is better to make a copy of the resource and then update the copy, then replace the original resource with the copy.

71 - Misusing `sync.WaitGroup`

Add must be called in the parent goroutine before the child goroutine is started.
Done must be called in the child goroutine when the child goroutine is finished.
Wait must be called in the parent goroutine after all child goroutines have been started.

72 - Forgetting about `sync.Cond`

sync.Cond is used to signal a condition to multiple goroutines.
The updater goroutine locks a mutex, updates the condition, unlocks the mutex, and then signals the condition.
The listener goroutine(s) lock the mutex, wait for the condition to be signaled, and then unlock the mutex.
- Lock/unlock is done to offset the fact that Wait unlocks the mutex and then relocks it when the condition is signaled.
If no goroutines are waiting for the condition when the call to Broadcast occurs, the signal is lost.

73 - Not using `errgroup`

errgroup provides tools to manage a group of goroutines and propagate errors.
It can be used to share context between goroutines to allow for cancellation of the group of goroutines upon the first error. If 3 calls are made in parallel and one of them fails fast with an error, the other two will be cancelled instead of waiting.

74 - Copying a sync type

sync.* types should never be copied.
If we have a value receiver receiving a struct containing a Mutex, for instance, it will be copied when passed to a function. This will lead to a data race.
To avoid the issue with a value receiver, either use a pointer receiver or make the mutex field a pointer.
The same issue will present itself if a function takes a sync.* type as an argument or if a field in a struct received as a function argument is a sync.* type.
go vet can catch these types of issues.

The standard library

75 - Providing a wrong time duration

Always use the time.Duration API to provide a time duration. Passing any int64 value would work, but the function accepts nanoseconds, which will be confusing.

76 - `time.After` and memory leaks

Resources created by time.After (e.g., a channel) are released only when the timer expires.
Using time.After inside a loop will create a new channel each time the loop iterates, leading to a memory leak. Each call to time.After will consume about 200 bytes. If this is a long-running loop with frequent events, this can be a significant memory leak (e.g., 5 million calls in an hour would be about 1 GB of wasted memory).
Use time.NewTimer instead, which will create a timer that can be reset, then listen for the timer to expire on the channel (<-timer.C).

77 - Common JSON-handling mistakes

When unmarshaling JSON, the json package will not set the fields of a struct to their zero values if the fields are not present in the JSON.
Type embedding in a struct can be problematic if the embedded type implements the json.Marshaler interface, which by virtue of being promoted will be used instead of the json.MarshalJSON method of the struct.
- Type embedding makes the parent struct implements interfaces of the embedded type.
If possible, use a named field instead of an anonymous field (type embedding) to avoid this issue.
Monotonic clocks guarantee that the time will always increase, even if the system clock is changed. This is useful for measuring time intervals.
- In Go, monotonic time is part of the same API that provides wall time.
- time.Equal does not take into account the monotonic clock, so it is possible for two times to be equal even if they are not.
- One can also strip away the monotonic time with time.Truncate(0).
To prevent time zone issues, use time.LoadLocationto load the desired time zone or usetime.UTC` to work with UTC time.
Using a map of any to unmarshal JSON may be convenient but has its drawbacks. For instance, numeric values will be unmarshaled as float64, even if they contain no decimals.

78 - Common SQL mistakes

“Forgetting that sql.Open doesn’t necessarily establish connections to a database”
- The behavior depends on the SQL driver.
- To make sure the connection is open, one should call Ping on the connection or use PingContext.
“Forgetting about connections pooling”
- SetMaxOpenConns should be set in any production-grade application to match the load the database can handle.
- If an application generates significant concurrent requests, the default value for SetMaxIdleConns (2) should be updated.
- If an application can face bursts of requests, one should set SetConnMaxIdleTime accordingly.
- SetConnMaxLifetime can be useful if an application is behind a load balancer, for example.
“Not using prepared statements”
- Prepared statements should be used in untrusted contexts.
“Mishandling null values”
- If the value of a column can be NULL, it should be structured as a pointer to receive nil values.
- Another approach is to use sql.NullString, sql.NullInt64, etc.
“Not handling row iteration errors”
- One should check rows.Err after iterating over rows to check for errors. This is because rows.Next can stop iterating over rows if an error occurs as well as when all rows have been read.

79 - Not closing transient resources

HTTP body
- The response body should be closed to avoid resource leaks (this is done automatically on the server side). This can be done conveniently with defer function that should call resp.Body.Close().
- The body should be closed even if it was not read (e.g., just checking the status code).
  - The body should be read to have a “keep-alive” connection (io.Discard is efficient for this purpose).
sql.Rows
- Rows should be closed to release the database connection.
os.File
- When closing a file in a defer function, the error should be sent back to the parent function to be handled in case of error while closing a writable file, for instance.
- If a write should be committed immediately, the file should be flushed before closing it with file.Sync() (this is a synchronous operation).

80 - Forgetting the return statement after replying to an HTTP request

http.Error does not stop the handler’s execution.
After calling http.Error, the handler should return to stop the execution.

81 - Using the default HTTP client and server

HTTP client
- The default client does not specify any timeouts.
HTTP server
- Just like the client, default timeouts should be overridden in production applications to avoid abuse and resource leaks.

Testing

82 - Not categorizing tests

They should be organized as per the testing pyramid:
- Unit tests: test a single function or method.
- Integration tests: test the interaction between multiple components.
- End-to-end tests: test the entire system.
It is not always necessary to run all tests at different stages of a project.
One can add a build flag such as // go:build integration to run integration tests only when needed.
- By default, files without a build flag will be run. Use go test --tags=integration -v . to run integration tests.
- It is possible to include negation in the build flag, e.g., // go:build !integration.
- Build tags apply to the whole file, not individual tests.
Using environment variables may be a better approach than build flags, because one can forget about existing tests that are being ignored and not explicitly flagged as such.
- Each test can then decide to check for environment variables being set and use t.skip to skip the test if the environment variable is not set.
Use testing.Short() in a conditional statement to skip long-running tests when running tests with the -short flag.

83 - Not enabling the `-race` flag

In Go, the race detector is a tool used at runtime to find data races.
- It is really useful, but brings quite a performance hit (5-10x memory, 2-20x execution time).
Concurrent code should be tested thoroughly to make use of the -race flag.
To avoid false positives in data race detections, one can use a loop to run the logic of the test multiple times.
If needed, // go:build !race can be used to exclude tests from running with the -race flag.

84 - Not using test execution modes

t.Parallel can be used to run tests in parallel.
- Sequential tests run first, then all parallel tests run at the same time (up to GOMAXPROCS, unless configured otherwise on the command-line).
There exists a -shuffle flag to randomize the order of tests, which isolates tests from each other.
- Using -v flag along with -shuffle will print the seed used to shuffle the tests, which can be useful to reproduce results from a CI environment.

85 - Not using table-driven tests

They are shorter and more readable than traditional tests.
They make refactoring tests easier.
They make test function names clearer/shorter.

86 - Sleeping in unit tests

Sleeping in a test can mean the test is flaky, especially when testing concurrent code.
Implementing a retry strategy is better than using a passive time.Sleep in a test.
- The testify package provides an Eventually function that can be used to retry a test function a number of times.
If a test can be synchronized, using a channel to receive the value from a goroutine to be tested can lead to deterministic results with no need for a sleep. This strategy can also implement a timeout using a select with a time.After case.

87 - Not dealing with the time API efficiently

Do not rely on time.Now in a function. If it must be tested, the time functionality could be injected as a dependency.
Even better, a function should allow receiving a time.Time as an argument, so that the caller can provide the needed value.
Time should also not be stored in a global variable, as all tests will share the same time, preventing them from running in parallel.
Whenever possible, testing time should be abstracted away from the code being tested.

88 - Not using testing utility packages

httptest package
- It can be used to test HTTP handlers.
- Instead of spinning up Docker, a fake server can be used to test the HTTP client.
iotest package
- io.Reader and io.Writer can be tested with this package.
- It can be used to test edge cases, such as reading a file that is too large or to make sure a reader is resilient against errors (e.g., network error).

89 - Writing inaccurate benchmarks

Use testing.B to run benchmarks. Then run go test -bench=., optionally adding the flag -benchtime=2s to run the benchmark for 2 seconds (default is 1 second).
Use b.ResetTimer after an expensive setup to reset the timer.
If the expensive call must be done for each iteration in the testing loop, then one can use b.StopTimer and b.StartTimer to stop and start the timer, respectively.
perflock can be used to lock the CPU frequency to avoid performance fluctuations during benchmarking.
Micro-benchmarks should be avoided, as they can be misleading. benchstat can be used to compare benchmarks. For instance, run go test -bench=. -count=10 | tee stats.txt to save the results to a file, then run benchstat stats.txt to compare the results.
Reusing the same variable in a benchmark can lead to incorrect results, as the compiler may optimize the code. To avoid this, use b.N to create a new variable for each iteration.
Forcing a benchmark to recreate data for each iteration of a loop may prevent CPU caching from affecting the results.

90 - Not exploring all the Go testing features

Use go test -coverprofile=coverage.out ./... in combination with go tool cover -html=coverage.out to generate a coverage report.
Test exposed behavior by creating a separate package so that unexported elements and internals are not tested.
testing.T can be used to test the behavior of a function, while testing.B can be used to test the performance of a function.
t.Cleanup can be used to clean up resources after a test (e.g., close a database connection).
When multiple cleanup functions are needed, they will be executed in reverse order of registration (LIFO), just like using defer.
TestMain can be used to set up and tear down resources for all tests in a package.

Optimizations

91 - Not understanding CPU caches

Mechanical sympathy is about understanding the hardware to write efficient code.
Most modern CPU architectures have three levels of cache: L1, L2, and L3.
L2 is about 4 times slower than L1, and L3 is about 10 times slower than L2.
L3 is a victim cache containing only data evicted from L2.
Access to main memory is between 50 and 100 times slower than access to L1 cache.
A cache line is the smallest unit of data that can be transferred between the cache and main memory. When first accessible a memory location, it results in a compulsory miss. Once the cache line is copied to the cache, subsequent accesses to the same memory location and contiguous ones will result in a cache hit.
A struct of slices can be about 20% faster than a slice of structs, because of spatial locality. For 16 elements, it may require 2 cache lines instead of 4 for a struct containing fields with slices.
Data structures affect predictability at the CPU level.
- Unit stride is when the CPU can predict the next memory location to access. All values are contiguous in memory.
- Constant stride is when the CPU can predict the next memory location to access (e.g., read every two elements of a slice), but the values are not contiguous in memory.
- Non-unit stride is when the CPU cannot predict the next memory location to access. The values are not contiguous in memory. This would be the case for a linked list or a slice of pointers.
Cache placement policy works roughly with the principle of least recently used (pseudo-LRU). When a cache line is full, the least recently used cache line is evicted.
Conflict misses happen because the cache is partitioned into sets, and two memory locations map to the same set. This can be avoided by aligning data structures to the cache line size.
A critical stride accesses memory addresses with the same set index, leading to a conflict miss.
This is why micro-benchmarking can be misleading: if the production system uses a different cache architecture, the results may be widely different.

False sharing is when two threads access different variables that are on the same cache line, and at least one of the goroutines is a writer.
This causes the cache line to be invalidated.
One solution is to add padding to the struct to ensure that the variables are on different cache lines, so that the same cache line is not copied across different logical cores.
Another approach is to use channels to communicate between goroutines, which will avoid false sharing, instead of having two goroutines share the same struct.

93 - Not taking into account instruction-level parallelism

Data hazards occur when an instruction depends on the result of a previous instruction (e.g., read and increment a variable).
Control hazards occur when the CPU cannot predict the next instruction to execute (e.g., conditionals).
Sometimes, introducing a temporary variable can help the CPU predict the next instruction and parallelize the execution.
One should remain cautious about micro-optimizations, since the Go compiler keeps evolving, and the generated assembly code may change.

94 - Not being aware of data alignment

"[…] a variable’s memory address should be a multiple of its own size": this is the principle of data alignment.
A struct size must be a multiple of the word size (8 bytes on 64-bit systems).
A struct being an atomic unit, it should be aligned to the word size, as it won’t be reorganized by the compiler. The compiler only adds padding to the fields to guarantee data alignment.
Rule of thumb: organize structs by size, from largest to smallest, to avoid padding. This leads to better spatial locality and cache usage.

95 - Not understanding stack vs. heap

Stack is the default memory. LIFO. Stores all local variables for a goroutine.
Currently, each goroutine initially gets 2 KB of stack space (contiguous memory, to preserve data locality), which can grow up to 1 GB.
A stack frame is created for each function call, and it is destroyed when the function returns. The stack frames pile up from accessing different functions inside a given goroutine.
A stack is self-cleaning: it requires no garbage collection. Each goroutine has its own stack.
Using pointers, slices, and maps will allocate memory on the heap. One reason is that memory addresses from a stack frame coming from a returned function could not be accessed anymore with a stack.
The memory heap in Go is managed by the garbage collector and is shared among all goroutines. Variables that could still be referenced after exiting a function are stored on the heap.
Allocations on the stack are trivial and faster than on the heap. The stack is also more cache-friendly.
Escape analysis is a compiler optimization that determines whether a variable can be allocated on the stack or the heap.
- “Sharing up” escapes to the heap (need a reference from a context that is about to be destroyed).
- “Sharing down” stays on the stack (no reference to the variable outside the function).
- Global variables are stored on the heap, because they can be shared across goroutines.
- A pointer sent to a channel will escape to the heap.
- A variable referenced by a value sent to a channel.
- When a local variable is too large to fit on the stack.
- When building a program, the -gcflags "-m" flag can be used to see the escape analysis results.

96 - Not knowing how to reduce allocations

The Reader interface takes in a slice of bytes and returns the number of bytes read and an error. If it had been the other way around, the slice would always escape to the heap.
sync.Pool can be used to reduce the number of allocations. It is a pool of objects that can be reused. E.g., when writing a slice of bytes to a buffer, the buffer can be reused.
sync.Pool is thread-safe.

97 - Not relying on inlining

Mid-stack inlining is when the compiler inlines a function call that is in the middle of the stack, not at the top (e.g., main calls foo, which calls bar, both functions are simple, so bar is inlined in foo and foo is in turn inlined in main).
Slow vs. fast paths can be optimized by the compiler. The slow path can be extracted to a separate function, which will be inlined in the fast path (“fast-path inlining technique”).

98 - Not using Go diagnostics tooling

Profiling
- With profiling, one can see where the program is spending most of its time. pprof is the Go profiler.
- There are different profiles: CPU (time spent), goroutine (stack traces of ongoing routines), heap (check memory leaks and current allocations), mutex (check time spent in locking calls), block (see where goroutines waits on synchronous operations).
- A profile can be generated while benchmarking: go test -bench=. -cpuprofile profile.out. This profile can be used with go tool pprof -http=:8080 profile.out.
- Heap profiling
  - It’s a good idea to force garbage collection before dumping the heap profile to avoid false assumptions. This can be done via the web UI at http://localhost/debug/pprof/heap?gc=1.
  - Snapshots can be compared with go tool pprof -http=:8080 -diff_base <file2> <file1>.
- Goroutine profiling
  - We can inspect to check for “goroutine leaks”.
- Block profiling
  - Useful to check performance affected by blocking calls. Must call runtime.SetBlockProfileRate to enable it. This profiler stays on until the program ends.
- Mutex profiling
  - Useful to check performance affected by mutex locks. Must call runtime.SetMutexProfileFraction to enable it. This profiler stays on until the program ends.
- A single profiler should be enabled at a time.
Execution tracer
- Used to check how garbage collection performs, how goroutines are scheduled, and how the scheduler behaves.
- It can be enabled with go test -bench=. -v -trace=trace.out or by downloading a trace file with /debug/pprof/trace?debug=0.
- The granularity of the tracer is per goroutine (unlike CPU profiling, which is per function).

99 - Not understanding how the GC works

debug.FreeOSMemory() can be used to force the garbage collector to release memory to the OS.
GODEBUG=gctrace=1 can be used to see the garbage collector in action.
One can force a large allocation of memory with something like var min = make([]byte, 1_000_000_000) so that the garbage collector will only trigger once the heap reaches twice the size of the allocation, which can reduce the impact on latency if otherwise many garbage collections would be triggered in a short period of time. On most systems, this leads to a lazy allocation (in the virtual address space). This is useful when the heap peak is known in advance.

100 - Not understanding the impacts of running Go in Docker and Kubernetes

CPU throttling can occur in Docker and Kubernetes, leading to performance degradation.
GOMAXPROCS should be set to the number of CPUs available to the container.

Resources and references

Book summary: Refactoring UI

Sébastien Lavoie — Sat, 09 Sep 2023 17:56:00 -0600

Introduction

It covers core principles like establishing consistent systems for recurring elements such as spacing and typography.
It advises generous use of white space and thoughtful visual hierarchy.
Steps for adding depth through lighting, shadows, and layers are outlined.
Recommendations are provided for working with images and illustrations effectively.
Tips on elevating default elements and breaking conventions to avoid typical, cookie-cutter designs are included.
Studying other designers’ work and unintuitive decisions is encouraged to expand one’s skills.
The goal is to equip readers with knowledge to thoughtfully craft interfaces that appear purposefully designed without simply mimicking common graphical tropes.
By learning fundamental visual design concepts tailored to user interfaces, anyone can create clean, functional, and visually engaging digital experiences.

Starting from scratch

Start with a feature, not a layout
- Design the shell and navigation later once you know what features you need.
Detail comes later
- Design with a big sharpie.
- Design in grayscale to use spacing, contrast and size to do the heavy lifting.
Don’t over-invest
- Go with low-fidelity sketches and wireframes first to move fast. Don’t get bogged down with details like fonts and colors early on.
Don’t design too much
- Work in cycles: design a feature simply, build it, fix errors.
- Be a pessimist: expect it to be hard to build, and postpone building it (e.g., commenting system without attachments at first).

Choose a personality

Font choice
- Elegant/classic: serif
- Playful: Sans-serif
- Plain: neutral sans-serif
Color
- Blue: safe and familiar, good overall pick.
- Gold: expensive, sophisticated.
- Pink: more fun, less serious.
Border radius
- Small – neutral, not much personality.
- Large – more playful.
- None – more serious/formal.
- Stay consistent – either rounded or not, same everywhere.
Language
- Choose the right words for the desired tone.
Decide what you actually want
- Look at the websites your expected customers use? Serious, then that’s the feel. Playful? Then that.

Limit your choices

Define systems in advance
- Have a working palette color and limit yourself to it.
- Use a type scale to avoid picking specific font sizes.
Designing by process of elimination.
Systematize as much as possible to avoid repetitive low-level decisions.

Hierarchy

Visual hierarchy refers to how elements appear in importance relative to one another.
Don’t let HTML semantics dictate visual styling – style based on visual hierarchy.
Size isn’t everything. Bold font communicates importance. De-emphasize secondary/tertiary information to highlight the most important elements.
Rely on font weight and color, not just font size, to create hierarchy.
Limit yourself to 2-3 colors and font weights to define hierarchy.
On colored backgrounds, pick a color with similar hue but lower saturation/lightness rather than just making text gray.
Avoid label: value formats – use formats, context, and combined labels/values for clarity without labels.
Treat labels as secondary content when needed – smaller, lower contrast, lighter weight.
De-emphasize competing elements instead of just emphasizing the key element.
Combine labels and values: e.g., instead of “In stock: 12”, use “12 left in stock”.
Treat labels as supporting content.
Design buttons based on hierarchy, not just semantics. Primary/destructive actions don’t always need prominence.

Layout and spacing

Start with too much white space, then remove until happy – often ends up being “just enough”.
Compact layouts have their place but should be a deliberate choice, not the default.
Establish a spacing and sizing system
- A linear scale won’t work
- Create a constrained spacing/sizing system based on a base unit (like 16px) to work faster with more consistency.
Don’t feel the need to fill the whole screen – give elements only the space they need.
Grids are overrated
- Not all elements should be fluid
- Don’t shrink an element until you need to.
- Grids can bring order but don’t have to dictate every layout decision.
Balance layouts in columns instead of just making things wider.
Relative sizing doesn’t scale.
- Fix element widths when flexibility isn’t needed.
- Scale elements independently – don’t rely on relative units like em.
Avoid ambiguous spacing: increase spacing between groups and reduce spacing within groups to show relationships.

Text

Establish a constrained type scale to speed up font size decisions and add consistency. Avoid strict modular scales.
Use pixel or rem units – not em – to guarantee sizes match the scale.
For UI design, pick scale values by hand instead of using a mathematical ratio.
Use high quality fonts – safe bets are neutral sans-serifs and system fonts.
Use the system stack if not sure: -apple-system, Segoe UI, Roboto, Noto Sans, Ubuntu, Cantarell, Helvetica Neue;
Favor fonts with 5+ weights and optimized for legibility.
Keep line length between 45-75 characters (20-35 em) for optimal readability.
Align mixed font sizes by their baselines, not centers.
Use taller line-heights for small text and shorter line-heights for large text.
Subtly emphasize links in UIs, no need for high contrast colors.
Left align most text, right align numbers in tables. Justify only with hyphenation.
Don’t center long form text.
Tighten letter-spacing for headlines set in text faces. Loosen letter-spacing for all-caps.
Not all links need to be emphasized boldly, especially in UIs where there is plenty of interactivity.

Colors

HSL (hue, saturation, lightness) is more intuitive to the human eye: use it instead of HEX and RGB.
- Hue is the color position on the color wheel. 0° is red, 120° is green, and 240° is blue.
- Saturation represents the vividness. 0% is grey while 100% is vibrant and intense.
- Lightness measures how close a closer is to black (0%) or white (100%), where 50% is the pure color.
HSL is not the same as HSB (lightness != brightness). In HSB, 0% brightness is always black but 100% brightness is white when saturation is set to 0%.
- With 100% saturation in both HSL and HSB, 100% brightness is equivalent to 50% lightness (pure color).
Don’t rely solely on color to convey information - support it with other indicators.

A good color palette

You need more colors than you think - aim for 10+ shades of greys, primary colors, and accent colors.
Grey
- There is a lot of grey in UIs: between 8-10 shades of grey is a good array of options to pick from. Use very dark grey instead of true black so it looks more natural.
Primary colors
- 1 or 2 is great, with 5-10 different shades (e.g., Facebook is basically blue). A very light shade is good for a tinted background (e.g., alerts), darker shades work great for text.
Accent colors
- Use sparingly.
- Needed to bring semantic meaning, such as red for danger, yellow for warnings and green for positive trends.
- For a complex UI, can be as many as 10 different colors, each with 5-10 shades.

Building a color palette

Define color shades upfront instead of generating on the fly. Pick base, darkest, lightest, then fill in gaps.
Don’t rely purely on math to generate shades - trust your eyes and tweak if needed.
Choose the base color first. This is the color in the middle that light and dark shades are derived from.
Center: pick a shade that would work well as a button background.
Extremes: pick the lightness color to work as a background and the darkest to work on top as text to get enough contrast.
Repeat the process to pick the remaining colors in between. 9 shades is a good sweet spot as it divides cleanly.
For grays, the base color doesn’t matter nearly as much: same process.

Perceived brightness

In the HSL system, the further away a color is from 50% lightness, the more saturated it should be so it doesn’t look washed out.
The perceived brightness of a color to the human eye can be calculated as follows:

brightness = √(0.299·r² + 0.587·g² + 0.114·b²) / 255

Green is perceived as more bright than red, which in turn is perceived as more bright than blue. With this data point, we can tweak the perceived brightness of a color by rotating the color wheel towards the desired perceived brightness.
Grays aren’t usually totally desaturated.
- To make them feel “cool” saturate them with some blue.
- To make them feel “warm”, saturate them with yellow or orange.
- The same “saturation curve” applies to grays: the further away from 50% brightness, the more they should be saturated to avoid looking washed out.

A word on accessibility

Instead of having elements like dark-colored pills with a white foreground color, flip the contrast by having light-colored pills with a dark version of the color in the foreground so they won’t grab too much attention.
Rotate the hue to make the color pop more. For instance, instead of cranking up the lightness to meet ideal contrast ratios, rotate the hue and possibly keep the saturation high for a vibrant contrast.
Don’t rely on color alone
- Add visual cues (like arrows for upward/downward trends). Designing in grayscale first will help spot these kinds of issues.
- For things like a graph, playing with contrast will produce clearer results.

Depth

Mimic how light works in the real world.
- Emulate a light source coming from above to make elements appear raised or inset.
- For instance, with a button, if light comes from above, then the top should be lighter (e.g., box-shadow inset) than the bottom (box-shadow, casting shadow only below).
- For elements that are “inset” (I.e., more depth in the center), the opposite effect should happen: more light reaches the bottom part (vertical box-shadow, going up) than the top part (box-shadow inset such that it casts some shadow towards the bottom).
Don’t overdo it.

Shadows

Use shadows to convey elevation
- When something appears to be raised from the background (e.g., bigger drop-shadow), it attracts the user’s focus.
- Small shadows are useful to attract attention to the primary action to be performed.
- Medium shadows are useful for things like a dropdown so it looks separate from the background.
- Large shadows are great for modals.
- 5 options is usually plenty.
- Can be used for interactivity, like when dragging a row from a table.
- A button can feel like it’s being “pressed” by reducing its shadow on click (i.e., it gets closer to the page or farther away from the user).
- Casting shadows
  - To simulate a shadow cast by direct light (e.g., a long shadow behind a plant getting some sunlight), use a larger, more diffuse and subtle shadow.
  - To simulate a shadow cast by ambient light (e.g., the shadow right below the plant’s pot where light can’t hardly reach), use a narrow and darker shadow.
  - Apply both types of shadow casting at once to create visually compelling effects.
Define a fixed set of shadows as an elevation system for consistency.
Even flat designs can have depth
- Flat design does not convey depth with shadows and gradients.
- Create depth with color: lighter colors feel closer.
- Can use short, offset shadows with no blur at all.
Overlap elements to create layers
- E.g., position a block with a negative margin so it appears on top of two different sections.
- For smaller elements like avatars, an invisible border around it can make it pop better.

Images and contrast

Images
- Use high quality photos - hire a pro or use stock sites for generics. Don’t expect good results from phone pics.
Text needs consistent contrast
- Add an overlay to background images to get readable headlines.
- Lower the image contrast. Adjust the brightness to compensate.
- Colorize the image with a single color.
- Use text shadows instead of overlays to preserve image dynamics.

Scale

Don’t scale up icons
Even for SVGs, they will lack detail if they were designed correctly at a smaller scale: use a more detailed icon instead.
If updating the icon is not an option, enclose it inside another shape (e.g., a circle with a background color) to occupy more space.
Don’t scale down screenshots
Details will be distorted and hard to read.
Either use a presentation from a smaller device (e.g., the tablet version on a desktop) or take a partial screenshot (e.g., “zoom in” effect with a circle or simply crop the original image).
If a more complex UI needs to be presented, simplify the original image so the user doesn’t try to read a 4px font size but instead sees simpler shapes.
Don’t scale down icons
E.g. for a favicon, redraw a simplified version instead: details will look fuzzy at a small size.
User-submitted content (e.g., Instagram feed)
Control the size and shape: crop images to keep the aspect ratio of containers so they don’t disturb layout (e.g., background-size: cover).
Add inner shadows or borders to user images to prevent background bleed.

Finishing touches

Use icons instead of bullets in a bulleted list.
Play with size and colors. E.g., you can have huge quotes around the block of text representing a testimonial.
Links deserve special styling.
Use custom checkboxes and radio buttons in forms: integrate the brand’s primary color(s).
Add color with accent borders, e.g. across the top of a card element, when highlighting active elements in the UI or to the left of alert messages.
Decorate backgrounds
- Change the background color. Works great for different page sections.
- Can use gradients: use two hues that are within 30˚ of each other.
- Use repeating patterns.
Consider the empty state carefully, especially when the user is meant to fill the UI with data.
- Prioritize empty states with illustrations and clear calls to action.
- I.e., a contact page with no contacts should feel welcoming, with some icon and button to add a contact instead of a blank page with an error-like statement.
- Similarly, things like dashboard may present a bunch of options that won’t apply until there is something to show: simplify the user experience.
Use less borders
- Use box shadows, spacing, and color instead of borders to distinguish elements.
- Have two background colors for containers (e.g., darker footer that looks separate from the main content).
- Replace borders with additional spacing.
Make components more exciting
- E.g., a table may not need all its columns if a particular column does not need to be sorted and could instead be part of another one (e.g., combining a dollar amount with a plan type or a person’s name with its title).
- Radio buttons could be cards instead if they are an important part of the UI.

Next level

Look for unintuitive decisions in designs you admire to find new ideas.
Rebuild interfaces you love to discover small details that make them exceptional.
Study others to build a solid foundation.

Conclusion

The concepts covered in this book demonstrate that effective visual design does not require innate artistic talent or formal training. By approaching interfaces methodically, establishing constraints through systems for recurring elements, and learning to strategically apply principles like visual hierarchy, white space, and depth cues, anyone can create designs that appear polished and purposeful. While internalizing these fundamentals does take practice, the ideas presented remove much of the subjective, nuanced expertise needed for things like picking complementary colors or balancing composition in illustrations.

By focusing on interfaces rather than free-form graphical work, the required skills become more structured and intentional choices drive outcomes. While studying formally can provide more breadth, the core principles in this text are enough to equip non-designers with the ability to build clean, intuitive, and aesthetically pleasing digital products. By implementing the recommendations covered, engineers and others new to design can gain confidence in their interface design skills. A great, concise read!

Book summary: The Effective Engineer

Sébastien Lavoie — Sun, 16 Apr 2023 14:49:00 -0600

Introduction

This wonderful, mostly non-technical book is a compilation of the best practices and lessons learned by the author, Edmond Lau, during his impressive career as a software engineer. It is a must-read for anyone who wants to improve their skills and become more effective at what they do. The following are short excerpts from the condensed knowledge found in the book, sprinkled with some minor comments of my own along the way. These are extracted from the book’s takeaway sections and headings: I wholeheartedly recommend reading the work in its entirety for an immersive exposure to the fantastic storytelling capabilities of Edmond!

Book summary

Part 1: Adopt the Right Mindset

Focus on high-leverage activities

Leverage = Impact produced / Time invested

Leverage is the ROI for the effort put in.
Meta-skills are important to know where to focus one’s time and energy to translate more of one’s efforts into impact.
Leverage your time by always finding ways to perform activities quicker and making sure those activities will have a meaningful impact.

Optimize for learning

Make your learning rate a priority: it compounds like interest over time.
Study code for core abstractions written by the best engineers at your company.
Write more code.
Go through any technical, educational material available internally.
Master the programming languages that you use.
Send your code reviews to the harshest critics.
Enroll in classes on areas where you want to improve.
Participate in design discussions of projects you’re interested in.
Work on a diversity of projects.
Make sure you’re on a team with at least a few senior engineers whom you can learn from.
Jump fearlessly into code you don’t know.
Learn new programming languages and frameworks.
Invest in skills that are in high demand.
Read books.
Join a discussion group.
Attend talks, conferences, and meetups.
Build and maintain a strong network of relationships.
Follow bloggers who teach.
Write to teach.
Tinker on side projects.
Pursue what you love.

Prioritize regularly

Track to-dos in a single list.
Focus on what produces value directly.
Focus on the important and non-urgent.
Limit the amount of work in progress.
Use implementation intentions (if-then planning).
Prioritize routinely.
Write down and review to-dos.
Reduce context switches.

Part 2: Execute, Execute, Execute

Invest in iteration speed

Move fast to learn fast.
Invest in time-saving tools.
Shorten your debugging and validation loops.
Master your programming environment.
Get proficient with your favorite text editor or IDE.
Learn at least one productive, high-level programming language (scripting).
Get familiar with UNIX (or Windows) shell commands.
Prefer the keyboard over the mouse.
Automate your manual workflows.
Test out ideas on an interactive interpreter.
Make it fast and easy to run just the unit tests associated with your current changes.
Don’t ignore non-engineering bottlenecks.

Measure what you want to improve

Use metrics to drive progress.
Pick the right metric to incentivize the behavior you want.
Instrument everything to understand what is going on.
Internalize useful numbers (e.g., number of active users, requests per second, amount of data accessed and written daily, etc.).
Be skeptical about data integrity.
Log data liberally, in case it turns out to be useful later on.
Build tools to iterate on data accuracy sooner.
Write end-to-end integration tests to validate your entire analytics pipeline.
Examine collected data sooner.
Cross-validate data accuracy by computing the same metric in multiple ways.
When a number does look off, dig into it early.

Validate your ideas early and often

Find low-effort ways to validate your work.
- Approach a problem iteratively to reduce wasted effort.
Continuously validate product changes with A/B testing.
- Reduce the risk of large implementations by using small validations.
Beware the one-person team (you need feedback early to make sure you’re working on the right thing!).
Be open and receptive to feedback.
Commit code early and often.
Request code reviews from thorough critics.
Ask to bounce ideas off your teammates.
Design the interface or API of a new system first.
Send out a design document before devoting your energy to your code.
If possible, structure ongoing projects so that there is some shared context with your teammates.
Solicit buy-in for controversial features before investing too much time.
Build feedback loops for your decisions.

Improve your project estimation skills

Project estimation is one of the hardest skills that an effective engineer needs to learn. But it’s crucial to master.
To produce accurate estimates:
- Decompose the project into granular tasks.
- Estimate based on how long tasks will take, not on how long you or someone else wants them to take.
- Think of estimates as probability distributions, not best-case scenarios.
- Let the person doing the actual task make the estimate.
- Beware of anchoring bias (e.g., hearing about a low estimate from someone else may skew our own estimate later to be too low).
- Use multiple approaches to estimate the same task (e.g., estimate from past experiences, decompose a larger task into smaller ones and estimate each one).
- Beware the mythical man-month. Having more people on a project doesn’t necessarily mean it will be completed faster. Communication overhead, context switching, and other factors can slow down the team as much as $O(n^2)$ considering the number of people involved.
- Validate estimates against historical data.
- Use time-boxing to constrain tasks that can grow in scope.
- Allow others to challenge estimates.
Allow buffer room for the unknown in the schedule.
Define measurable milestones.
Do the riskiest tasks first.
Know the limits of overtime.

Part 3: Build Long-Term Value

Balance Quality with Pragmatism

Establish a culture of reviewing code.
- Catch bugs or design shortcomings early.
- Increase accountability for code changes.
- Provide an avenue for sharing best practices.
- Increase long-term agility.
Invest in good software abstractions to simplify difficult problems.
- Reduce the complexity of the original problem into easier-to-understand primitives.
- Reduce future application maintenance and make it easier to apply future improvements.
- DRY principle: solve the hard problems once and re-use the solutions multiple times.
Scale code quality with automated testing.
- Make large refactors with confidence.
- Offer executable documentation.
- Don’t test everything: focus on the most important parts of the code and make sure everything critical is covered.
Manage your technical debt.
- Pay it periodically. Incur it when necessary to meet deadlines.
- Repay the debts with the highest interest rates first (i.e., focus on what produces the most leverage).

Minimize Operational Burden

Embrace operational simplicity.
- Increased complexity introduces more potential single points of failure.
- New engineers face a steeper learning curve when learning and understanding the new systems.
- Effort towards improving abstractions, libraries, and tools gets diluted across the different systems.
- Do the simple thing first.
Build systems to fail fast.
- Pinpoint the source of errors.
- Make debugging easier by not masking your errors and by not deferring failures until later.
Relentlessly automate.
- Don’t underestimate the future frequency of the task.
- Internalize the time savings over a long time horizon.
- Automate mechanics over decision-making.
Make batch processes idempotent.
- At least, make processes “retryable” if they cannot be made idempotent.
Hone your ability to respond and recover quickly.
- Plan and practice failure modes.

Invest in Your Team’s Growth

Make hiring everyone’s responsibility.
- Identify which qualities you care about the most in your colleagues: keep the bar high.
- Make sure the recruiting process is effective.
- Adapt interview problems to different levels of difficulty based on the candidate’s experience.
Design a good onboarding process.
- Ramp up new engineers as quickly as possible (e.g., with codelabs).
  - Help the people around you be successful.
- Impart the team’s culture and values (e.g., have onboarding talks).
- Expose new engineers to the breadth of fundamentals needed to succeed (e.g., with mentorship).
- Socially integrate new engineers onto the team (e.g., give them starter tasks to feel they’re part of the team faster).
Share ownership of code (increase the bus factor).
- Avoid one-person teams.
- Review each other’s code and software designs.
- Make sure the team is exposed to different types of tasks and responsibilities regularly.
- Focus on readable and high-quality code.
- Share software decisions and architecture clearly.
- Make sure code and workflows are documented properly.
- Invest in teaching and mentoring.
  - The more effective the team becomes, the more freedom there is to work on new projects.
Build collective wisdom with post-mortems (e.g., at NASA, these become the user manual).
Build a great engineering culture.
- Optimize for iteration speed.
- Always automate.
- Use the right software abstractions.
- Nurture a respectful working environment.
- Allot experimentation time.
- Foster a culture of learning (CI/CD: continuous improvement, continuous delightfulness).
- Hire the best!

Conclusion

Edmond opened my eyes to the importance of being an effective engineer, showing what that entails and how to concretely achieve it. His extraordinary technical journey and knowledge of the field are evident throughout the book, and I found myself nodding in agreement with many of his points. He made me think about how I could apply his advice to my own work. Notably, the key concept of leverage resonated deeply with me and ended up being the cornerstone of the final project I built for my bachelor’s degree: ProductiviDo - An efficient task planner focusing on high-leverage activities.

Resources and references

Edmond Lau - The author’s personal website.
ProductiviDo - A simple and efficient todo-app built with React-Native.
The Effective Engineer: How to Leverage Your Efforts In Software Engineering to Make a Disproportionate and Meaningful Impact - goodreads.com.
The Effective Engineer - by Edmond Lau.

sglavoie.com – Tag: Skimming Notes

Reading notes: A Philosophy of Software Design, 2nd Edition

Nature of complexity

Strategic vs. tactical programming

Modules should be deep

Information hiding

General-Purpose Modules are Deeper

Different Layer, Different Abstraction

Pull Complexity Downwards

Better Together Or Better Apart?

Define Errors Out Of Existence

Design it Twice

Why Write Comments? The Four Excuses

Comments Should Describe Things that Aren’t Obvious from the Code

Choosing names

Use comments as part of the design process

Modifying existing code

Consistency

Code should be obvious

Software trends

Designing for performance

Decide what matters

Conclusion

Book summary: 100 Go Mistakes and How to Avoid Them

Go: Simple to learn but hard to master

Go outline

Simple doesn’t mean easy

100 Go mistakes

Code and project organization

1 - Unintended variable shadowing

2 - Unnecessary nested code

3 - Misusing init functions

4 - Overusing getters and setters

5 - Interface pollution

6 - Interface on the producer side

7 - Returning interfaces

8 - any says nothing

9 - Being confused about when to use generics

10 - Not being aware of the possible problems with type embedding

11 - Not using the functional options pattern

12 - Project misorganization

13 - Creating utility packages

14 - Ignoring package name collisions

15 - Missing code documentation

16 - Not using linters

Data types

17 - Creating confusion with octal literals

18 - Neglecting integer overflows

19 - Not understanding floating points

20 - Not understanding slice length and capacity

21 - Inefficient slice initialization

22 - Being confused about nil vs. empty slices

23 - Not properly checking if a slice is empty

24 - Not making slice copies correctly

25 - Unexpected side effects using slice append

26 - Slices and memory leaks

27 - Inefficient map initialization

28 - Maps and memory leaks

29 - Comparing values incorrectly

Control structures

30 - Ignoring the fact that elements are copied in range loops

31 - Ignoring how arguments are evaluated in range loops

32 - Ignoring the impact of using pointer elements in range loops

33 - Making wrong assumptions during map iterations

34 - Ignoring how the break statement works

35 - Using defer inside a loop

Strings

36 - Not understanding the concept of a rune

37 - Inaccurate string iteration

38 - Misusing trim functions

39 - Under-optimized string concatenation

40 - Useless string conversions

41 - Substrings and memory leaks

Functions and methods

42 - Not knowing which type of receiver to use

43 - Never using named result parameters

44 - Unintended side effects with named result parameters

45 - Returning a nil receiver

46 - Using a filename as a function input

47 - Ignoring how defer arguments and receivers are evaluated

8 - `any` says nothing

25 - Unexpected side effects using slice `append`

35 - Using `defer` inside a loop

71 - Misusing `sync.WaitGroup`

72 - Forgetting about `sync.Cond`

73 - Not using `errgroup`

76 - `time.After` and memory leaks

83 - Not enabling the `-race` flag