sglavoie.com – Category: Learnings

Reading notes: A Philosophy of Software Design, 2nd Edition

Sébastien Lavoie — Sun, 30 Mar 2025 11:18:00 -0600

Nature of complexity

“Change amplification: The first symptom of complexity is that a seemingly simple change requires code modifications in many different places.” (e.g., refactoring a project to inject dependencies in tests, because the system was hard to test in the first place)
“Cognitive load: The second symptom of complexity is cognitive load, which refers to how much a developer needs to know in order to complete a task.” (e.g., close a file where it is opened whenever possible, instead of leaking context and pushing the responsibility onto the caller)
“Unknown unknowns: The third symptom of complexity is that it is not obvious which pieces of code must be modified to complete a task, or what information a developer must have to carry out the task successfully.”
“One of the most important goals of good design is for a system to be obvious.”

Strategic vs. tactical programming

“it’s worth taking a little extra time to find a simple design for each new class; rather than implementing the first idea that comes to mind, try a couple of alternative designs and pick the cleanest one. Try to imagine a few ways in which the system might need to be changed in the future and make sure that will be easy with your design”

Modules should be deep

Modular design
- “In general, if a developer needs to know a particular piece of information in order to use a module, then that information is part of the module’s interface.”
Abstractions
- “The key to designing abstractions is to understand what is important, and to look for designs that minimize the amount of information that is important”
A deep module is one in which the interface is small and the implementation is large.
- “Deep modules such as Unix I/O and garbage collectors provide powerful abstractions because they are easy to use, yet they hide significant implementation complexity.”
Shallow modules
- “A shallow module is one whose interface is complicated relative to the functionality it provides.”

Information hiding

“If a piece of information is hidden, there are no dependencies on that information outside the module containing the information, so a design change related to that information will affect only the one module”
Information leakage
- “If the affected classes are relatively small and closely tied to the leaked information, it may make sense to merge them into a single class.”
- “When designing modules, focus on the knowledge that’s needed to perform each task, not the order in which tasks occur.”
- “it’s important to avoid exposing internal data structures as much as possible”
“The best features are the ones you get without even knowing they exist.”
“As a software designer, your goal should be to minimize the amount of information needed outside a module; for example, if a module can automatically adjust its configuration, that is better than exposing configuration parameters”
“think about the different pieces of knowledge that are needed to carry out the tasks of your application, and design each module to encapsulate one or a few of those pieces of knowledge. This will produce a clean and simple design with deep modules.”

General-Purpose Modules are Deeper

“specialization leads to complexity”
Generality leads to better information hiding
“Eliminate special cases in code”

Different Layer, Different Abstraction

“A pass-through method is one that does nothing except pass its arguments to another method, usually with the same API as the pass-through method. This typically indicates that there is not a clean division of responsibility between the classes.”

Pull Complexity Downwards

Figure out complexity in the library instead of pushing it up to the user.
"[…] you should avoid configuration parameters as much as possible. Before exporting a configuration parameter, ask yourself: will users (or higher-level modules) be able to determine a better value than we can determine here?"

Better Together Or Better Apart?

“Bring together if information is shared”
“Bring together if it will simplify the interface”
“Bring together to eliminate duplication”
- Use goto to exit nested code blocks early as long as it’s not too complex.
On Splitting long methods: “If you make a split of this form and then find yourself flipping back and forth between the parent and child to understand how they work together, that is a red flag (“Conjoined Methods”) indicating that the split was probably a bad idea.”
“If the caller has to invoke each of the separate methods, passing state back and forth between them, then splitting is not a good idea.”

Define Errors Out Of Existence

“The key overall lesson from this chapter is to reduce the number of places where exceptions must be handled; in some cases the semantics of operations can be modified so that the normal behavior handles all situations and there is no exceptional condition to report […].”
“The best way to eliminate exception handling complexity is to define your APIs so that there are no exceptions to handle: define errors out of existence”
E.g., the Java substring method could remove its IndexOutOfBoundsException by automatically rounding the index to the nearest valid value.
"Exception masking doesn’t work in all situations, but it is a powerful tool in the situations where it works. It results in deeper classes, since it reduces the class’s interface (fewer exceptions for users to be aware of) and adds functionality in the form of the code that masks the exception. Exception masking is an example of pulling complexity downward."
“The idea behind exception aggregation is to handle many exceptions with a single piece of code; rather than writing distinct handlers for many individual exceptions, handle them all in one place with a single handler.” (e.g., a web server dispatching different types of requests and catching all sorts of errors).
“In most applications there will be certain errors that are not worth trying to handle. Typically, these errors are difficult or impossible to handle and don’t occur very often. The simplest thing to do in response to these errors is to print diagnostic information and then abort the application.” (e.g., out-of-memory errors).

Design it Twice

“The initial design experiments will probably result in a significantly better design, which will more than pay for the time spent designing it twice”
“The process of devising and comparing multiple approaches will teach you about the factors that make designs better or worse. Over time, this will make it easier for you to rule out bad designs and hone in on really great ones.”

Why Write Comments? The Four Excuses

Good code is self-documenting. Comments serve as precisions on abstractions.
I don’t have time to write comments. The benefits of having documentation quickly offset the cost of writing it.
Comments get out of date and become misleading. Keep the documentation close to where it needs to be, so you update it.
All the comments I have seen are worthless. Be the change you want to see.
Benefits of well-written comments
- “The overall idea behind comments is to capture information that was in the mind of the designer but couldn’t be represented in the code.”
- “One of the purposes of comments is to make it it unnecessary to read the code.”

Comments Should Describe Things that Aren’t Obvious from the Code

Don’t repeat the code
“Lower-level comments add precision”
- Focus on what the variable represents, not how it is modified (the latter is explained by the code).
“Higher-level comments enhance intuition”
- “What is the simplest thing you can say that explains everything in the code? What is the most important thing about this code?”
Interface documentation
“Implementation comments: what and why, not how”
- Describe at a more abstract level to help understand what is happening and why. The code tells the how.
Cross-module design decisions
- One possible solution is to use a designNotes.md file or similar to jot down all major decisions, and point to it from the code (e.g., See topic in designNotes).

Choosing names

Great names are “precise, unambiguous, and intuitive.”
Use names consistently: make it so that a concept is expressed in the same words regardless of the context in which it is found if it makes sense to do so.
Avoid extra words: no more Hungarian Notation with modern IDEs.

Use comments as part of the design process

Delayed comments are bad comments
- You won’t remember exactly what you had in mind and why you did things if you delay writing documentation: design decisions will get lost.
Write the comments first
- No backlog of unwritten comments.
- “it produces better comments.”
Comments are a design tool
- Writing them at the beginning “improves the system design.”
- “If a method or variable requires a long comment, it is a red flag that you don’t have a good abstraction.”
Early comments are fun comments
- “If you are programming strategically, where your main goal is a great design rather than just writing code that works, then writing comments should be fun, since that’s how you identify the best designs.”
- “I’m looking for the design that can be expressed completely and clearly in the fewest words. The simpler the comments, the better I feel about my design.”
Are early comments expensive?
- “Writing the comments first will mean that the abstractions will be more stable before you start writing code.”
- “In contrast, if you write the code first, the abstractions will probably evolve as you code, which will require more code revisions than the comments-first approach.”

Modifying existing code

“Whenever you modify any code, try to find a way to improve the system design at least a little bit in the process. If you’re not making the design better, you are probably making it worse.”
Maintaining comments: keep the comments near the code
- “The farther a comment is from its associated code, the less likely it is that it will be updated properly.”
- “In general, the farther a comment is from the code it describes, the more abstract it should be (this reduces the likelihood that the comment will be invalidated by code changes).”
Comments belong in the code, not the commit log
- “An example is a commit message describing a subtle problem that motivated a code change. If this isn’t documented in the code, then a developer might come along later and undo the change without realizing that they have re-created a bug.”
Maintaining comments: avoid duplication
Maintaining comments: check the diffs
- Review changes during the pre-commit phase to make sure documentation stays up-to-date with the code changes.
Higher-level comments are easier to maintain
- Most helpful comments do not repeat the code, they provide an abstraction that can’t be expressed easily by the code alone.

Consistency

Similar things are done in a similar way.
“Consistency creates cognitive leverage: once you have learned how something is done in one place, you can use that knowledge to immediately understand other places that use the same approach.”
Ensuring consistency
- Document most important conventions.
- “the value of consistency over inconsistency is almost always greater than the value of one approach over another.”

Code should be obvious

“the best way to determine the obviousness of code is through code reviews”
“If someone reading your code says it’s not obvious, then it’s not obvious, no matter how clear it may seem to you”
Things that make code more obvious
- Good names
- Consistency
- Judicious use of white space
- Blank lines
- Comments
Things that make code less obvious
- Event-driven programming
- Generic containers
- Different types for declaration and allocation

Software trends

Object-oriented programming and inheritance
- “In order for an interface to have many implementations, it must capture the essential features of all the underlying implementations while steering clear of the details that differ between the implementations; this notion is at the heart of abstraction.”
- “Class hierarchies that use implementation inheritance extensively tend to have high complexity.”
- “Before using implementation inheritance, consider whether an approach based on composition can provide the same benefits.”
Agile development
- May encourage a more tactical approach to programming. Balance is required.
Test-driven development
- “it focuses attention on getting specific features working, rather than finding the best design”
- “One place where it makes sense to write the tests first is when fixing bugs. Before fixing a bug, write a unit test that fails because of the bug.”
Design patterns
- “As with many ideas in software design, the notion that design patterns are good doesn’t necessarily mean that more design patterns are better.”
Getters and setters
- “It’s better to avoid getters and setters (or any exposure of implementation data) as much as possible.”

Designing for performance

How to think about performance
- “The key is to develop an awareness of which operations are fundamentally expensive.”
Measure before (and after) modifying
- “the goal is to identify a small number of very specific places where the system is currently spending a lot of time, and where you have ideas for improvement.”
- “If the changes didn’t make a measurable difference in performance, then back them out (unless they made the system simpler).”
Design around the critical path
- “When redesigning for performance, try to minimize the number of special cases you must check.”
“Complicated code tends to be slow because it does extraneous or redundant work.”

Decide what matters

Minimize what matters
How to emphasize things that matter
- “important things should appear in places where they are more likely to be seen, such as interface documentation, names, or parameters to heavily used methods.”
- “repetition: key ideas appear over and over again.”
- “The things that matter the most should be at the heart of the system, where they determine the structure of things around them.”
Mistakes
- “treat too many things as important. When this happens, unimportant things clutter up the design, adding complexity and increasing cognitive load.”
- “fail to recognize that something is important. This mistake leads to situations where important information is hidden, or important functionality is not available so developers must continually recreate it.”
Thinking more broadly
- “the best way to make a document easy to read is to identify a few key concepts at the beginning and structure the remainder of the document around them. When discussing the details of a system, it helps to tie them back to the overall concepts.”
- “Focusing on what is important is also a great life philosophy: identify a few things that matter most to you, and try to spend as much of your energy as possible on those things.”

Conclusion

“The reward for being a good designer is that you get to spend a larger fraction of your time in the design phase, which is fun. Poor designers spend most of their time chasing bugs in complicated and brittle code.”

Book summary: 100 Go Mistakes and How to Avoid Them

Sébastien Lavoie — Sat, 24 Aug 2024 14:55:00 -0600

Go: Simple to learn but hard to master

Go outline

Go does not have:
- Type inheritance;
- Macros;
- Partial functions;
- Support for lazy variable evaluation;
- Support for immutability;
- Operator overloading;
- Pattern matching;
- …
Go is:
- Stable;
- Expressive;
- Fast to compile;
- Safe (type-safety);
- Simple;
- Excellent for concurrent programming (goroutines, channels).

Simple doesn’t mean easy

Concurrency in Go may be simpler than in other languages, but it is still hard to get right, as per this study.

100 Go mistakes

Learning from mistakes is efficient.
“the best time for brain growth is when we’re facing mistakes” (source).
“we can remember not only the error but also the context surrounding the mistake” (source).
Categories of mistakes
- Bugs
  - Bugs are very costly (source).
  - Bugs can be very dangerous (e.g., in the case of a radiation therapy machine).
- Needless complexity
  - Don’t over-complicate things.
- Weaker readability
  - “the ratio of time spent reading versus writing is well over 10 to 1” (source).
  - Nested code.
  - Data type representation.
  - Not using named result parameters.
- Suboptimal or unidiomatic organization
  - Utility packages.
  - Init functions.
- Lack of API convenience
  - Overusing any types.
  - Wrong creational pattern.
  - Blindly applying OOP principles.
- Under-optimized code
  - Lack of fundamental knowledge.
  - Impacted performance.
  - Floating-point accuracy.
  - Poorly parallelized execution.
  - Not knowing how to reduce allocations.
  - Data alignment.
- Lack of productivity
  - Know how to write efficient tests.
  - Use the standard library.
  - Use profilers and linters.

Code and project organization

1 - Unintended variable shadowing

A variable name declared in a block can be re-declared in a nested block (variable shadowing).

 go
 
 
 
// client is shadowed
var client *http.Client // will always be nil
 if tracing {
 client, err := createClientWithTracing()
 if err != nil {
 return err
 }
 log.Println(client)
 } else {
 client, err := createDefaultClient()
 if err != nil {
 return err
 }
 log.Println(client)
 }

Solutions
- Use a temporary variable name in the inner scope.
- Use the = operator instead of := in the inner scope. Leads to a single assignment. Error handling can be handled in the outer scope (client, err = ...).
- Be caution with reusing err as well.

2 - Unnecessary nested code

Put the happy path first and return early on all edge cases and errors.
Instead of using else blocks, use early returns.
Instead of checking for an happy path, check for the error path and return where possible.

3 - Misusing init functions

Refresher on init functions
- An init function is a special function that is executed before the main function. It takes no arguments and returns no values.
- Variables are initialized, the init function is called, and then the main function is called.
- Imported packages are initialized before the main function is called and before the init function in the main package is called because the main package depends on the imported packages.
- Packages can define multiple init functions, in which case they are executed in alphabetical order based on the source code file name. Hence, we should not depend on the order of execution of init functions as file names can change.
- The same source file can define multiple init functions, in which case they are executed in the order they are defined in the file.
- The underscore operator (_) can be used to import a package without using it. This is useful to execute the init function of a package without using the package itself.
- init functions cannot be invoked explicitly.
When to use init functions
- They should not be used to initialize variables, as error handling is limited (they don’t return errors, one must panic).
- When testing a package, the init function is executed before the tests are run.
- If side effects are achieved via variables, these become global variables, which pollute the global namespace. Any function in the package can access these variables, which can lead to unexpected behavior. Unit tests depending on such functions will be harder to write.
- The responsibility of error handling should be left up to the caller.
- They can be helpful to define static configurations, as is done in the init function of the Go blog.

4 - Overusing getters and setters

Go has no support for automatic getters and setters, and it is idiomatic to use direct access to fields.
When the use case justifies it, a getter should be named FieldName (not GetFieldName) and a setter should be named SetFieldName.

5 - Interface pollution

Go’s interfaces are satisfied implicitly (structural typing).
“The bigger the interface, the weaker the abstraction.” — Rob Pike
When to use interfaces
- Common behavior
  - The behavior is factored out into an interface when it is common to multiple types (e.g., sort.Interface).
- Decoupling
  - The implementation is decoupled from the interface, which allows for easier testing and swapping of implementations.
- Restricting behavior
  - The interface restricts the behavior of the type, which can be useful for enforcing constraints. For example, the io.Reader interface restricts the behavior of the type to reading. It is possible to enforce constraints such as with a getter interface that only has a Get method and prevents setting values.
Interface pollution
- Abstractions should be discovered, not invented. Only create interfaces when they are needed.
- Adding levels of indirection makes the code harder to understand.

6 - Interface on the producer side

Producer side refers to an interface defined in the same package as the type that satisfies it.
Consumer side refers to an interface defined in a package that uses it and which is independent of the package that implements the interface.
The client should determine the level of abstraction it needs.
This relates to the Interface segregation principle of SOLID.

7 - Returning interfaces

In many cases, this is a bad practice in Go.
“Be conservative in what you do, be liberal in what you accept from others.” — Transmission Control Protocol
- Return structs instead of interfaces.
- Accept interfaces if possible.
Interfaces might be returned (e.g., io.Reader), but this should be done with caution. It’s usually done when the interface itself is implemented on the producer side (e.g., standard library) and the consumer side (e.g., user code) is expected to use it.

8 - `any` says nothing

The interface{} type is a placeholder for any type. any is an alias for interface{} since Go 1.18.
It requires a type assertion to be used.
Using any does not convey any meaningful information about the type.
It will lead to issues compilation-wise.

9 - Being confused about when to use generics

A “type parameter” is a placeholder for a type that is specified when the function is called.
A constraint is a condition that the type parameter must satisfy (e.g., implement an interface like comparable).

 go
 
 
 
type intStringConstraint interface {
 ~int | ~string
 // ^--- union operator
}

The union operator (|) is used to specify that the type parameter must be either int or string.
The tilde (~) is used to restrict all the underlying types of the type parameter to the specified types.
Common uses and misuses
- Generics are useful to factor out common behavior (e.g, the sort interface).
  - Data structures (e.g., binary tree, linked list, heap).
  - Functions working with slice, map, channels of any type.
- Generics are not useful when the behavior is not common.
  - If the method of the type argument is called, then the generic should be removed in the function received and a type should be specified in the function parameter.
- Generics do not shine when they make the code more complex. Unnecessary abstraction should be avoided.

10 - Not being aware of the possible problems with type embedding

A type embedding is a way to reuse the fields and methods of a type in another type. The fields from that other type get “promoted” to the new type (i.e., they can be accessed directly from the new type, as well as through the embedded type).
Incorrect usage
- An example of a wrong use of type embedding is a sync.Mutex embedded in a struct. It will be available outside the struct and can be misused.
- It shouldn’t be used only as syntactic sugar to access the methods of the embedded type. Use a field instead.
- It shouldn’t promote fields that are not intended to be used outside the struct (e.g., sync.Mutex).
Correct usage
- A custom logger that wants to write and close may embed a io.WriteCloser interface so that both methods are available. They will be promoted to the new type and there won’t be a need to implement these methods simply to forward a call to the embedded type.
- Preventing type embeddings in public structs is probably the safest route.
Embedding vs. OOP subclassing
- Embedding is not subclassing. It is a way to reuse fields and methods of a type in another type. It is a form of composition.

11 - Not using the functional options pattern

Config struct
- This is limited because to bring flexibility and distinguish between the zero value and the default value, one must use pointers.
Builder pattern
- To keep chaining method calls, one should not return an error, which means validation is delayed.
- To have proper error handling along the way, it makes the configuration more complex than it needs to be.
Functional options pattern
- This pattern makes use of closures to set the configuration options, where the closure is an anonymous function being returned.
- The closure is called with the configuration struct as an argument, and it sets the configuration options.
- This pattern allows for validation of the configuration options and for the configuration to be set in a single call.
- It is a flexible pattern that allows for the configuration to be set in any order and for the configuration to be validated at the time of setting.
- One does not need to provide a default value for the configuration struct since the function accepts variadic arguments, which may be empty.

12 - Project misorganization

Project structure
- The project-layout is a great place to start for a complex application. Despite not being an official standard, it is a great and logically sound starting point.
Package organization
- There is no concept of subpackages in Go. Sub-directories are useful to have high cohesion.
- Hexagonal architecture group per technical layer.
- An alternative is to group by context (e.g., customer context, contract context).
- Avoid both nano and huge packages.
  - Nano packages probably mean there’s lack of understanding about the connections between the different files.
  - Huge packages dilute the meaning of the package.
  - Name a package after what they provide, not what they contain.
  - “Package names should be short, concise, expressive and, by convention, a single lowercase word”.
- Minimize what should be exported as much as possible to reduce coupling.

13 - Creating utility packages

utils, common, base… bad practice.
util is meaningless.
While nano packages should be avoided, if a few functions are highly cohesive and don’t belong anywhere else, it’s a good idea to keep them separate with good naming.
Make APIs expressive.

14 - Ignoring package name collisions

Naming a variable the same as a package name will shadow the package name.
Either rename the variable (redisClient := redis.NewClient()) or rename the import with an alias (import redisapi "mylib/redis").
The same is true for built-in functions like copy: they shouldn’t be shadowed.

15 - Missing code documentation

Simplifies how clients consume an API.
Helps in maintaining a project.
In Go, every exported element should be documented, starting with the name of the element being documented (e.g., Customer is ...). It should be a complete sentence ending with a period.
An element can be marked as deprecated with a comment starting with Deprecated: after the description of the element.
The convention to document a package is to have a comment at the top of the file with the package name and a description of the package, starting with // Package packageName .... The first line should be concise, then a blank line, then a more detailed description.
Comments at the top of the file will be omitted from the documentation if they are followed by a blank line and more comments.

16 - Not using linters

Automatic tool to analyze code and catch errors.
They help prevent mistakes automatically and consistently.

Data types

17 - Creating confusion with octal literals

Using 0o instead of 0 for octal literals is clearer.
Octal is still useful to express file permissions (e.g., 0644).
Go supports using underscores in numeric literals to improve readability (e.g., 1_000_000).

18 - Neglecting integer overflows

It won’t panic at runtime. They are silent bugs.
It will wrap around.
Use the math package to check for overflows.
Detecting integer overflow when incrementing
- Use math.MaxInt, math.MinInt, math.MaxUint, math.MinUint.
Detecting integer overflows during addition
- Use math.MaxInt - a < b or math.MaxUint - a < b.
Detecting integer overflows during multiplication
- Use math.MaxInt/a < b or math.MaxUint/a < b.

19 - Not understanding floating points

A mantissa is the significant part of a floating-point number.
The exponent is the power of 10 by which the mantissa is multiplied.
For float32, the mantissa is 23 bits and the exponent is 8 bits.
For float64, the mantissa is 52 bits and the exponent is 11 bits, the remaining bit is the sign bit.
It is best to avoid comparing by equality with floating-point numbers and instead use a tolerance (delta).
Use math.IsInf and math.IsNaN to check for infinity and NaN.
Group floating-point numbers together by their magnitude to reduce precision loss when adding and subtracting.
With calculations involving both addition/subtraction and multiplication/division, it is best to perform the multiplication/division first to reduce precision loss.

20 - Not understanding slice length and capacity

Slice data is stored contiguously in an array data structure.
A slice holds a pointer to the array, the length of the slice, and the capacity of the slice.
Trying to access an index greater than the length of the slice will result in a runtime panic, even though the capacity is greater.
Slicing a slice will create a new slice with the same underlying array, but with a different length and capacity.
The append function will create a new slice with a new underlying array if the capacity is exceeded.

21 - Inefficient slice initialization

Use make to initialize a slice with a specific length and capacity.
Set a length of 0 (2nd argument to make) and a capacity of n (3rd argument to make), which will reuse the same backing array until the capacity is exceeded.
One can also set only the length and use direct assignment instead of append (bars[i] = fooToBar(foo)). This is about 400% faster than using an empty slice.
If the length of the slice is already known, it is better to take advantage of it and set the length of the slice when initializing it.

22 - Being confused about nil vs. empty slices

A slice is empty if its length is equal to 0.
A slice is nil if it equals nil.
A nil slice is an empty slice.
A nil slice can be passed in a single line: s := append([]int(nil), 42).
A nil slice is marshaled as a null element, whereas a non-nil, empty slice is marshaled as an empty array.

23 - Not properly checking if a slice is empty

"[…] when designing interfaces, we should avoid distinguishing nil and empty slices, which leads to subtle programming errors".
“This principle is the same with maps. To check if a map is empty, check its length, not whether it’s nil”.
Do len(slice) != 0 instead of slice != nil.

24 - Not making slice copies correctly

The destination slice must have the same length as the source slice (or greater). Else, the copy will match the length of the destination slice and may be missing elements.
copy is the more idiomatic way to copy slices (vs. dst := append([]int(nil), src...)).

25 - Unexpected side effects using slice `append`

If the resulting slice as a length smaller than the capacity, the slice will be modified in place and will have unintended side effects.
To avoid side effects outside a given range of a slice, full slice expression can be used (e.g., s := []int{1, 2, 3}, then s[:2:2] will create a new slice with a length of 2 and a capacity of 2).

26 - Slices and memory leaks

Leaking capacity
- Slicing a huge slice to a smaller slice will keep the capacity of the original slice. This makes matters worst if the result of a slice is kept for many iterations in a loop.
- Instead, use copy to create a new slice with the necessary length.
Leakage with full slice expressions
- The issue is the same as with slicing.
- The solution is the same as with slicing.
Slice and pointers
- A slice holds a pointer to an array, so if the slice is kept, the array will be kept in memory.
- To avoid this, use copy to create a new slice with the necessary length.
- If however we want to keep the original capacity, we can set all the unneeded elements to nil explicitly. For instance, all elements but the first 2:

 go
 
for i := 2; i < len(foos); i++ {
 foos[i].v = nil
 }

27 - Inefficient map initialization

Maps are unordered collections of key-value pairs, where all keys are unique. They use hash tables under the hood.
Insertion can be an O(n) operation if the map is resized.
make can be used to initialize a map with a specific size (second argument to make, without capacity). This avoids having to grow the map and re-balancing all the buckets of 8 elements each.

28 - Maps and memory leaks

The number of buckets in a map cannot shrink. Removing elements from a map will not reduce the number of buckets.
Even the overflown buckets will not be reduced.
One possible solution: periodically copy the map to a new map.
- Downside: the amount of memory used will be doubled for a short period of time.
Another solution: update the value type of the map to store an array pointer.
- The peak memory consumed will be far less.
- The garbage collection will be more efficient.

29 - Comparing values incorrectly

== does not work for slices, maps, and functions.
reflect.DeepEqual could be used, but it’s about 100x slower than ==, and so might be better used for testing only.
If strict comparison is needed on structs and good performance is required, consider using a custom comparison function.

Control structures

30 - Ignoring the fact that elements are copied in range loops

In Go, every assignment leads to a copy of the value.
In a range loop, the value is copied, not the reference.
To update a slice with a range loop, one must use the index to update the slice, i.e., for i := range s { s[i] = ... }.
A classic for loop can be used to update the slice in place, i.e., for i := 0; i < len(s); i++ { s[i] = ... }.
Using a pointer to a slice will allow for the slice to be updated in place, but iterating over such a slice is less efficient for the CPU.

31 - Ignoring how arguments are evaluated in range loops

Looping over an array will copy the value of the array element.
Using a pointer to the array will allow for the array to be updated in place and will avoid copying values, which may be more efficient, especially on large arrays.
The range loop evaluates the expression to its right only once, so if the expression is a slice, the slice will not be updated in the loop.
To access updated elements of a slice during iteration with a range loop, one must use the index to access the element in the slice, not the value of the element, since the expression is copied to a temporary variable.

32 - Ignoring the impact of using pointer elements in range loops

Using a pointer to a slice when looping with a range loop can lead to storing the address of the temporary variable in the slice for all elements in the slice.
To avoid this issue, one can create a local variable for each element to be iterated over in the slice and then use the local variable in the loop to perform assignments.
Another solution is to use a range loop with the index, create a temporary variable by dereferencing the pointer, and then assign the temporary variable to the slice.

33 - Making wrong assumptions during map iterations

Maps don’t sort keys.
Insertion order is not preserved.
Iteration order is unspecified. This is a conscious language design choice so that developers don’t rely on the order of the keys.
One should work on a copy of a map if new elements are to be added or removed during iteration.

34 - Ignoring how the break statement works

break works on the innermost loop, whether that be a for, switch, or select.
To break outside of a loop, one can use a label.
A label can be an idiomatic way to break out of nested loops, as is done in the net/http package with a readlines: label.

35 - Using `defer` inside a loop

defer is a statement that schedules a function to be called when the surrounding function returns. That is, it won’t run on each iteration of the loop until it returns, if ever.
To avoid resource leaks like this, each iteration of the loop can use a separate function that is run and defers the cleanup.
Another solution is to use a closure, but this can be less readable.
A function call will decrease performance slightly, so if performance is a concern, the defer call should be handled manually before the loop.

Strings

36 - Not understanding the concept of a rune

A charset is a set of characters.
An encoding describes how to translate a character set into a sequence of bytes.
In Go, a string is a reference to an immutable slice of bytes.
A rune is a Unicode code point.
Source code in Go is encoded in UTF-8: string literals will then also be encoded in UTF-8.
UTF-8 is a variable-length encoding, where each code point is represented by one to four bytes. For instance, the string hello is represented by five bytes, but the string containing the character 汉 is represented by three bytes, which we can check with len("汉").
A rune is an alias for int32 (4 bytes = 32 bits).
The repository golang.org/x contains extensions to work with UTF-16 and UTF-32.

37 - Inaccurate string iteration

With the unicode/utf8 package, one can use utf8.RuneCountInString(s) to count the number of runes in a string.
Accessing specific runes can be done without converting the string if characters are in the ASCII range.
It is most useful to convert a string to a slice of runes when a specific rune is required by index.
Iterating over the runes with a for range loop returns the starting index of the rune and the rune itself.
Iterating with a for range using only the index returns the starting index of the rune and the byte value at that position.

38 - Misusing trim functions

Use TrimRight to remove trailing characters. Go backwards until a rune is not in the set of characters to be trimmed.
Use TrimSuffix to remove a suffix (only one substring occurrence).
Use TrimLeft to remove leading characters. Go forwards until a rune is not in the set of characters to be trimmed.
Use TrimPrefix to remove a prefix (only one substring occurrence).

39 - Under-optimized string concatenation

+= is inefficient for string concatenation because it creates a new string each time.
Use strings.Builder for efficient string concatenation.
Call Grow on the builder to allocate memory for the string.

40 - Useless string conversions

Working with []byte is more efficient than working with strings due to many interfaces in the standard library taking []byte as input, which avoids doing a conversion.
Most methods in the strings package have a []byte counterpart in the bytes package.

41 - Substrings and memory leaks

A substring shares the same backing array as the original string.
If the substring is kept, the original string will be kept in memory.
To avoid this, either use string([]byte(theString[:n])) to slice the needed part of the string or strings.Clone to create a new string.

Functions and methods

42 - Not knowing which type of receiver to use

Go does not pass values by reference.
A value receiver makes a copy of the value: changes are local to the method.
A pointer receiver passes the memory address of the value: changes are global to the value. Strictly speaking, this is still a copy, but the copy is made for a pointer.
Use a pointer receiver when:
- The method needs to mutate the receiver.
- The method receiver contains a field that cannot be copied.
- The receiver is a large struct.
Use a value receiver when:
- Enforcing immutability.
- The receiver is a map, func or chan.
- A slice that doesn’t have to be mutated.
- The receiver is a small struct or array with no mutable fields, like time.Time.
- The receiver is a basic type, like int, float64, or string.
Mixing value and pointer receivers can lead to confusion and bugs: this should generally be avoided (although the standard library shows some exceptions, like time.Time).
Defaulting to a value receiver is a good practice, unless there is a good reason to use a pointer receiver.

43 - Never using named result parameters

Only use named result parameters when they make the code more readable (e.g., returning two int can be ambiguous).
Use naked returns only when the function is short and the return values are clear.
For an interface definition, named result parameters are useful to document the return values.

44 - Unintended side effects with named result parameters

Named result parameters are initialized to their zero value.
Using named result parameters doesn’t mean one has to use naked returns: clarity is more important than brevity.

45 - Returning a nil receiver

A nil receiver is a pointer to a struct that has not been initialized.
Instead of returning a pointer to a struct (which will return a non-nil pointer to a nil struct), return nil directly.

46 - Using a filename as a function input

It is more idiomatic and flexible to use an io.Reader or io.Writer as an input to a function that reads or writes to a file: this abstracts the data source.
This allows for the function to be used with any type that implements the io.Reader or io.Writer interface, not just a file.
It makes testing easier, as one can pass in a bytes buffer instead of a file, for instance.

47 - Ignoring how defer arguments and receivers are evaluated

The arguments to a defer statement are evaluated when the defer statement is executed, not when the deferred function is executed.
- That means that if a variable is initialized with a zero value and passed to defer as is, the deferred function will receive the zero value.
- To avoid this pitfall, one can use a pointer to the variable, which will be evaluated when the deferred function is executed. The memory address will not change, but the value at that address will.
- Another solution is to use a closure to capture the value of the variable at the time the defer statement is executed.
  - This works because the values of the variables are evaluated when the closure runs, not when the closure is created.
In the case of a method receiver, data in the receiver is evaluated when the defer statement is executed, leading to the same situation as with arguments.
- A pointer receiver can be used to avoid this issue, as the updated value of the pointer will be visible to the deferred function when it is executed.

Error management

48 - Panicking

It is used to signal genuinely unexpected errors, such as programming errors (e.g., net/http checking for a status code outside the valid range).
Another use case for panic is when the application requires a dependency but fails to initialize it (e.g., a regular expression).

49 - Ignoring when to wrap an error

%w is used to wrap an error conveniently.
The %v directive will print the transformed error message, but %w will print the error message and the wrapped error message.
If a caller relies on unwrapping an error provided with %w, it leads to coupling: if the underlying implementation that is wrapped changes, the caller will break.
To mark an error, one should use a custom error type.

50 - Checking an error type inaccurately

errors.As is used to check if an error is of a specific type anywhere in the error chain.
errors.As should be used whenever error wrapping is used.

51 - Checking an error value inaccurately

A sentinel error is a predefined error that can be compared to an error value. It is about an “expected” error (e.g., sql.ErrNoRows and io.EOF).
Unexpected errors should be implemented as error types, implementing the error interface.
errors.Is is used to check if an error is equal to a sentinel error (not ==! if the error is wrapped, it will not be equal to the sentinel error).

52 - Handling an error twice

Logging the same error twice can lead to confusion.
Logging an error is handling an error! One should not both log and return an error.
Error wrapping is useful to provide context to an error when propagating it up the call stack.

53 - Not handling an error

The only way to explicitly ignore an error is to assign it to the blank identifier _. Adding a comment about why the error is ignored is a good practice.
Do not simply ignore the assignment.

54 - Not handling defer errors

Errors should be handled or explicitly ignored in defer functions.
To return an error to the caller via the defer function, one can use a named return parameter, assigning the error to it.
Instead of returning multiple errors from a defer function, it may be a good idea to log one error and return the other, depending on the context.

Concurrency: Foundations

55 - Mixing up concurrency and parallelism

Concurrency is about structuring a program, while parallelism is about running multiple tasks at the same time.
Concurrency enables parallelism.

56 - Thinking concurrency is always faster

A thread is the smallest unit of execution that can be scheduled by an operating system.
A goroutine is a lightweight thread managed by the Go runtime: it is context-switched on and off by an OS thread.
A goroutine has a memory footprint of 2 KB, while a thread has a memory footprint of ~2 MB.
There is a point of diminishing returns when creating too many goroutines: one must take into account the overhead of context switching and determine the size of the workload that will be processed by each goroutine.
It is often best to start with a sequential solution and use profiling tools to determine where to add concurrency.

57 - Being puzzled about when to use channels or mutexes

Synchronization between parallel goroutines is achieved with mutexes (mutex stands for mutual exclusion).
Channels are used to communicate between goroutines. If there are 3 goroutines and a third one needs results from the first two, a channel is the way to go.
Mutexes ensure exclusive access to a resource (e.g., a map).
Coordination or ownership of a resource is achieved with channels.
In general, mutexes are needed for parallel goroutines, whereas channels are needed for concurrent ones.

58 - Not understanding race problems

A data race occurs when two goroutines access the same variable concurrently and at least one of the accesses is a write.
sync/atomic provides atomic operations to avoid data races. It works only on primitive types.
Channels can be used to perform atomic operations on non-primitive types, then send the result over the channel.
Mutexes can be used to protect a critical section of code.
The absence of a data race doesn’t mean the code will be deterministic: it is possible for goroutines to be scheduled in different orders, leading to a race condition.
A buffered channel can lead to a data race because a receive from an unbuffered channel happens before the send, but a send to a buffered channel happens before the receive.

59 - Not understanding the concurrency impacts of a workload type

A worker pool can be based on GOMAXPROCS (the number of OS threads) to efficiently distribute the workload when a workload is CPU-bound.
When a workload is I/O-bound, the number of goroutines can be increased to handle more I/O operations concurrently, depending on the external systems.

60 - Misunderstanding Go contexts

A context is a way to pass deadlines, cancellation signals, and other request-scoped values across API boundaries and between processes.
context.Background is the root context, which is never canceled, has no values, and has no deadline, used when there is no parent context.
Deadline
- A deadline is a point in time when the context should be canceled, specified as either a time.Time or a time.Duration (time from now).
- Deferring the cancellation of a context can prevent a goroutine from being blocked indefinitely or leaking resources.
Cancellation signals
- context.WithCancel returns a copy of the parent context with a new Done channel.
Context values
- context.WithValue returns a copy of the parent context with a key-value pair.
A function that users wait for should take a context as an argument.
Prefer context.TODO() to context.Background() when a context is needed but there is no parent context, as it better represents the intent.

Concurrency: Practice

61 - Propagating an inappropriate context

Propagating a context is about passing a context from one goroutine to another.
A context should be passed to a goroutine carefully: if the context requires some values to be passed but the parent goroutine gets its context cancelled, the child goroutine will also be cancelled. This may be undesirable.
- To avoid this, create a custom struct implementing a context that never expires and that does not carry a cancellation signal, so the values can be passed without the risk of being cancelled.

62 - Starting a goroutine without knowing when to stop it

A goroutine might loop over the values received from a channel, but if the channel is never closed, this will be a resource leak.
Passing a context to a goroutine (e.g., to expect a file to be closed) is bad design. Instead, the parent goroutine should be responsible for closing the file by calling a defer function right after the resource used by another routine is created.

63 - Not being careful with goroutines and loop variables

Spinning up a goroutine inside a for loop and relying on the loop variable can lead to unexpected behavior.
Either assign the value of the loop variable to a new variable inside the loop or pass the loop variable as an argument to the goroutine. In the first case, the goroutine could be set up as a closure. In the second case, it would be a regular function call on an anonymous function.

64 - Expecting deterministic behavior using select and channels

The select statement is used to wait on multiple channels. If multiple channels are ready, one will be chosen semi-randomly.
Using an unbuffered channel with a select statement will block until a value is sent on the channel.
Using a single channel and then checking the message type is an option.
Using a default case in a select statement will allow the program to continue if a channel no longer receives values.
- This approach works well for multiple channels: when a select case is chosen, add an inner select statement to handle the channel. For instance, after a disconnection message, handle all remaining messages sent the another channel by looping over the channel inside that select case.

65 - Not using notification channels

A notification channel is a channel that is used to signal the completion of a task.
If no data is needed to be sent, a struct{} can be sent over the channel. This is clearer than sending a bool or int. An empty struct occupies no memory.

66 - Not using nil channels

Sending a message to a nil channel will block forever.
Receiving from a closed channel is a non-blocking operation: the zero value of the channel type will be returned.
When looping over multiple channels and using select, if one of the channels is closed, this will lead to a busy-wait loop.
- E.g., say we have two channels in a function that are to be “merged” such that data returned from that function comes from the two channels. If one of the channels is closed, the function will still keep looping over the now invalid select case.
To avoid this, one can use a nil channel to signal a channel is closed. The select case will be skipped if the channel is nil, avoiding the busy-wait loop.

67 - Being puzzled about channel size

An unbuffered channel (synchronous channel) has a size of 0: the sender blocks until the receiver receives data from the channel.
With a buffered channel, the sender will block only when the buffer is full.
An unbuffered channel enables synchronization between goroutines.
An unbuffered channel can be used as a notification channel via a channel closure (close).
When using worker pools, a buffered channel can be used to limit the number of tasks that can be processed concurrently, matching the number of goroutines in the pool.
Rate-limiting can be achieved with a buffered channel, where the buffer size is the number of tasks that can be processed concurrently.
In a buffered channel, the lower the value, the bigger the impact on CPU. The higher the value, the bigger the impact on memory.
Buffered channels can lead to obscure deadlocks.
It’s usually best to start with a channel of size 1 and perform benchmarks to determine the optimal size.
Queues tend to be close to full or close to empty anyways (source).

68 - Forgetting about possible side effects with string formatting

Printing the values in a context will evaluate the values at the time of the call, which could lead to a data race.
Acquiring a lock only when it’s required is best. This reduces side effects and possible deadlocks. This ties nicely with the fact that a function can go over the “error cases” before doing anything else that might require more resources (e.g., check the age of a customer is valid before acquiring a lock).

69 - Creating data races with append

To prevent data races with append, one can make a copy of the slice in each goroutine before appending to it.
If the slice is initialized in such a way that the backing array is shared between goroutines, there will not be a data race since the original slice will remain unchanged.
Updating a map from one goroutine will lead to a data race if another goroutine tries to read from the map at the same time, given that the hashing algorithm for maps introduces some randomness.

70 - Using mutexes inaccurately with slices and maps

A mutex is used to protect a critical section of code from being accessed by multiple goroutines at the same time.
A mutex is used to protect a shared resource, like a slice or a map.
If an operation is lightweight, it is better to use a mutex to protect the resource (protecting the whole function in the process).
If an operation is heavy, it is better to make a copy of the resource and then update the copy, then replace the original resource with the copy.

71 - Misusing `sync.WaitGroup`

Add must be called in the parent goroutine before the child goroutine is started.
Done must be called in the child goroutine when the child goroutine is finished.
Wait must be called in the parent goroutine after all child goroutines have been started.

72 - Forgetting about `sync.Cond`

sync.Cond is used to signal a condition to multiple goroutines.
The updater goroutine locks a mutex, updates the condition, unlocks the mutex, and then signals the condition.
The listener goroutine(s) lock the mutex, wait for the condition to be signaled, and then unlock the mutex.
- Lock/unlock is done to offset the fact that Wait unlocks the mutex and then relocks it when the condition is signaled.
If no goroutines are waiting for the condition when the call to Broadcast occurs, the signal is lost.

73 - Not using `errgroup`

errgroup provides tools to manage a group of goroutines and propagate errors.
It can be used to share context between goroutines to allow for cancellation of the group of goroutines upon the first error. If 3 calls are made in parallel and one of them fails fast with an error, the other two will be cancelled instead of waiting.

74 - Copying a sync type

sync.* types should never be copied.
If we have a value receiver receiving a struct containing a Mutex, for instance, it will be copied when passed to a function. This will lead to a data race.
To avoid the issue with a value receiver, either use a pointer receiver or make the mutex field a pointer.
The same issue will present itself if a function takes a sync.* type as an argument or if a field in a struct received as a function argument is a sync.* type.
go vet can catch these types of issues.

The standard library

75 - Providing a wrong time duration

Always use the time.Duration API to provide a time duration. Passing any int64 value would work, but the function accepts nanoseconds, which will be confusing.

76 - `time.After` and memory leaks

Resources created by time.After (e.g., a channel) are released only when the timer expires.
Using time.After inside a loop will create a new channel each time the loop iterates, leading to a memory leak. Each call to time.After will consume about 200 bytes. If this is a long-running loop with frequent events, this can be a significant memory leak (e.g., 5 million calls in an hour would be about 1 GB of wasted memory).
Use time.NewTimer instead, which will create a timer that can be reset, then listen for the timer to expire on the channel (<-timer.C).

77 - Common JSON-handling mistakes

When unmarshaling JSON, the json package will not set the fields of a struct to their zero values if the fields are not present in the JSON.
Type embedding in a struct can be problematic if the embedded type implements the json.Marshaler interface, which by virtue of being promoted will be used instead of the json.MarshalJSON method of the struct.
- Type embedding makes the parent struct implements interfaces of the embedded type.
If possible, use a named field instead of an anonymous field (type embedding) to avoid this issue.
Monotonic clocks guarantee that the time will always increase, even if the system clock is changed. This is useful for measuring time intervals.
- In Go, monotonic time is part of the same API that provides wall time.
- time.Equal does not take into account the monotonic clock, so it is possible for two times to be equal even if they are not.
- One can also strip away the monotonic time with time.Truncate(0).
To prevent time zone issues, use time.LoadLocationto load the desired time zone or usetime.UTC` to work with UTC time.
Using a map of any to unmarshal JSON may be convenient but has its drawbacks. For instance, numeric values will be unmarshaled as float64, even if they contain no decimals.

78 - Common SQL mistakes

“Forgetting that sql.Open doesn’t necessarily establish connections to a database”
- The behavior depends on the SQL driver.
- To make sure the connection is open, one should call Ping on the connection or use PingContext.
“Forgetting about connections pooling”
- SetMaxOpenConns should be set in any production-grade application to match the load the database can handle.
- If an application generates significant concurrent requests, the default value for SetMaxIdleConns (2) should be updated.
- If an application can face bursts of requests, one should set SetConnMaxIdleTime accordingly.
- SetConnMaxLifetime can be useful if an application is behind a load balancer, for example.
“Not using prepared statements”
- Prepared statements should be used in untrusted contexts.
“Mishandling null values”
- If the value of a column can be NULL, it should be structured as a pointer to receive nil values.
- Another approach is to use sql.NullString, sql.NullInt64, etc.
“Not handling row iteration errors”
- One should check rows.Err after iterating over rows to check for errors. This is because rows.Next can stop iterating over rows if an error occurs as well as when all rows have been read.

79 - Not closing transient resources

HTTP body
- The response body should be closed to avoid resource leaks (this is done automatically on the server side). This can be done conveniently with defer function that should call resp.Body.Close().
- The body should be closed even if it was not read (e.g., just checking the status code).
  - The body should be read to have a “keep-alive” connection (io.Discard is efficient for this purpose).
sql.Rows
- Rows should be closed to release the database connection.
os.File
- When closing a file in a defer function, the error should be sent back to the parent function to be handled in case of error while closing a writable file, for instance.
- If a write should be committed immediately, the file should be flushed before closing it with file.Sync() (this is a synchronous operation).

80 - Forgetting the return statement after replying to an HTTP request

http.Error does not stop the handler’s execution.
After calling http.Error, the handler should return to stop the execution.

81 - Using the default HTTP client and server

HTTP client
- The default client does not specify any timeouts.
HTTP server
- Just like the client, default timeouts should be overridden in production applications to avoid abuse and resource leaks.

Testing

82 - Not categorizing tests

They should be organized as per the testing pyramid:
- Unit tests: test a single function or method.
- Integration tests: test the interaction between multiple components.
- End-to-end tests: test the entire system.
It is not always necessary to run all tests at different stages of a project.
One can add a build flag such as // go:build integration to run integration tests only when needed.
- By default, files without a build flag will be run. Use go test --tags=integration -v . to run integration tests.
- It is possible to include negation in the build flag, e.g., // go:build !integration.
- Build tags apply to the whole file, not individual tests.
Using environment variables may be a better approach than build flags, because one can forget about existing tests that are being ignored and not explicitly flagged as such.
- Each test can then decide to check for environment variables being set and use t.skip to skip the test if the environment variable is not set.
Use testing.Short() in a conditional statement to skip long-running tests when running tests with the -short flag.

83 - Not enabling the `-race` flag

In Go, the race detector is a tool used at runtime to find data races.
- It is really useful, but brings quite a performance hit (5-10x memory, 2-20x execution time).
Concurrent code should be tested thoroughly to make use of the -race flag.
To avoid false positives in data race detections, one can use a loop to run the logic of the test multiple times.
If needed, // go:build !race can be used to exclude tests from running with the -race flag.

84 - Not using test execution modes

t.Parallel can be used to run tests in parallel.
- Sequential tests run first, then all parallel tests run at the same time (up to GOMAXPROCS, unless configured otherwise on the command-line).
There exists a -shuffle flag to randomize the order of tests, which isolates tests from each other.
- Using -v flag along with -shuffle will print the seed used to shuffle the tests, which can be useful to reproduce results from a CI environment.

85 - Not using table-driven tests

They are shorter and more readable than traditional tests.
They make refactoring tests easier.
They make test function names clearer/shorter.

86 - Sleeping in unit tests

Sleeping in a test can mean the test is flaky, especially when testing concurrent code.
Implementing a retry strategy is better than using a passive time.Sleep in a test.
- The testify package provides an Eventually function that can be used to retry a test function a number of times.
If a test can be synchronized, using a channel to receive the value from a goroutine to be tested can lead to deterministic results with no need for a sleep. This strategy can also implement a timeout using a select with a time.After case.

87 - Not dealing with the time API efficiently

Do not rely on time.Now in a function. If it must be tested, the time functionality could be injected as a dependency.
Even better, a function should allow receiving a time.Time as an argument, so that the caller can provide the needed value.
Time should also not be stored in a global variable, as all tests will share the same time, preventing them from running in parallel.
Whenever possible, testing time should be abstracted away from the code being tested.

88 - Not using testing utility packages

httptest package
- It can be used to test HTTP handlers.
- Instead of spinning up Docker, a fake server can be used to test the HTTP client.
iotest package
- io.Reader and io.Writer can be tested with this package.
- It can be used to test edge cases, such as reading a file that is too large or to make sure a reader is resilient against errors (e.g., network error).

89 - Writing inaccurate benchmarks

Use testing.B to run benchmarks. Then run go test -bench=., optionally adding the flag -benchtime=2s to run the benchmark for 2 seconds (default is 1 second).
Use b.ResetTimer after an expensive setup to reset the timer.
If the expensive call must be done for each iteration in the testing loop, then one can use b.StopTimer and b.StartTimer to stop and start the timer, respectively.
perflock can be used to lock the CPU frequency to avoid performance fluctuations during benchmarking.
Micro-benchmarks should be avoided, as they can be misleading. benchstat can be used to compare benchmarks. For instance, run go test -bench=. -count=10 | tee stats.txt to save the results to a file, then run benchstat stats.txt to compare the results.
Reusing the same variable in a benchmark can lead to incorrect results, as the compiler may optimize the code. To avoid this, use b.N to create a new variable for each iteration.
Forcing a benchmark to recreate data for each iteration of a loop may prevent CPU caching from affecting the results.

90 - Not exploring all the Go testing features

Use go test -coverprofile=coverage.out ./... in combination with go tool cover -html=coverage.out to generate a coverage report.
Test exposed behavior by creating a separate package so that unexported elements and internals are not tested.
testing.T can be used to test the behavior of a function, while testing.B can be used to test the performance of a function.
t.Cleanup can be used to clean up resources after a test (e.g., close a database connection).
When multiple cleanup functions are needed, they will be executed in reverse order of registration (LIFO), just like using defer.
TestMain can be used to set up and tear down resources for all tests in a package.

Optimizations

91 - Not understanding CPU caches

Mechanical sympathy is about understanding the hardware to write efficient code.
Most modern CPU architectures have three levels of cache: L1, L2, and L3.
L2 is about 4 times slower than L1, and L3 is about 10 times slower than L2.
L3 is a victim cache containing only data evicted from L2.
Access to main memory is between 50 and 100 times slower than access to L1 cache.
A cache line is the smallest unit of data that can be transferred between the cache and main memory. When first accessible a memory location, it results in a compulsory miss. Once the cache line is copied to the cache, subsequent accesses to the same memory location and contiguous ones will result in a cache hit.
A struct of slices can be about 20% faster than a slice of structs, because of spatial locality. For 16 elements, it may require 2 cache lines instead of 4 for a struct containing fields with slices.
Data structures affect predictability at the CPU level.
- Unit stride is when the CPU can predict the next memory location to access. All values are contiguous in memory.
- Constant stride is when the CPU can predict the next memory location to access (e.g., read every two elements of a slice), but the values are not contiguous in memory.
- Non-unit stride is when the CPU cannot predict the next memory location to access. The values are not contiguous in memory. This would be the case for a linked list or a slice of pointers.
Cache placement policy works roughly with the principle of least recently used (pseudo-LRU). When a cache line is full, the least recently used cache line is evicted.
Conflict misses happen because the cache is partitioned into sets, and two memory locations map to the same set. This can be avoided by aligning data structures to the cache line size.
A critical stride accesses memory addresses with the same set index, leading to a conflict miss.
This is why micro-benchmarking can be misleading: if the production system uses a different cache architecture, the results may be widely different.

False sharing is when two threads access different variables that are on the same cache line, and at least one of the goroutines is a writer.
This causes the cache line to be invalidated.
One solution is to add padding to the struct to ensure that the variables are on different cache lines, so that the same cache line is not copied across different logical cores.
Another approach is to use channels to communicate between goroutines, which will avoid false sharing, instead of having two goroutines share the same struct.

93 - Not taking into account instruction-level parallelism

Data hazards occur when an instruction depends on the result of a previous instruction (e.g., read and increment a variable).
Control hazards occur when the CPU cannot predict the next instruction to execute (e.g., conditionals).
Sometimes, introducing a temporary variable can help the CPU predict the next instruction and parallelize the execution.
One should remain cautious about micro-optimizations, since the Go compiler keeps evolving, and the generated assembly code may change.

94 - Not being aware of data alignment

"[…] a variable’s memory address should be a multiple of its own size": this is the principle of data alignment.
A struct size must be a multiple of the word size (8 bytes on 64-bit systems).
A struct being an atomic unit, it should be aligned to the word size, as it won’t be reorganized by the compiler. The compiler only adds padding to the fields to guarantee data alignment.
Rule of thumb: organize structs by size, from largest to smallest, to avoid padding. This leads to better spatial locality and cache usage.

95 - Not understanding stack vs. heap

Stack is the default memory. LIFO. Stores all local variables for a goroutine.
Currently, each goroutine initially gets 2 KB of stack space (contiguous memory, to preserve data locality), which can grow up to 1 GB.
A stack frame is created for each function call, and it is destroyed when the function returns. The stack frames pile up from accessing different functions inside a given goroutine.
A stack is self-cleaning: it requires no garbage collection. Each goroutine has its own stack.
Using pointers, slices, and maps will allocate memory on the heap. One reason is that memory addresses from a stack frame coming from a returned function could not be accessed anymore with a stack.
The memory heap in Go is managed by the garbage collector and is shared among all goroutines. Variables that could still be referenced after exiting a function are stored on the heap.
Allocations on the stack are trivial and faster than on the heap. The stack is also more cache-friendly.
Escape analysis is a compiler optimization that determines whether a variable can be allocated on the stack or the heap.
- “Sharing up” escapes to the heap (need a reference from a context that is about to be destroyed).
- “Sharing down” stays on the stack (no reference to the variable outside the function).
- Global variables are stored on the heap, because they can be shared across goroutines.
- A pointer sent to a channel will escape to the heap.
- A variable referenced by a value sent to a channel.
- When a local variable is too large to fit on the stack.
- When building a program, the -gcflags "-m" flag can be used to see the escape analysis results.

96 - Not knowing how to reduce allocations

The Reader interface takes in a slice of bytes and returns the number of bytes read and an error. If it had been the other way around, the slice would always escape to the heap.
sync.Pool can be used to reduce the number of allocations. It is a pool of objects that can be reused. E.g., when writing a slice of bytes to a buffer, the buffer can be reused.
sync.Pool is thread-safe.

97 - Not relying on inlining

Mid-stack inlining is when the compiler inlines a function call that is in the middle of the stack, not at the top (e.g., main calls foo, which calls bar, both functions are simple, so bar is inlined in foo and foo is in turn inlined in main).
Slow vs. fast paths can be optimized by the compiler. The slow path can be extracted to a separate function, which will be inlined in the fast path (“fast-path inlining technique”).

98 - Not using Go diagnostics tooling

Profiling
- With profiling, one can see where the program is spending most of its time. pprof is the Go profiler.
- There are different profiles: CPU (time spent), goroutine (stack traces of ongoing routines), heap (check memory leaks and current allocations), mutex (check time spent in locking calls), block (see where goroutines waits on synchronous operations).
- A profile can be generated while benchmarking: go test -bench=. -cpuprofile profile.out. This profile can be used with go tool pprof -http=:8080 profile.out.
- Heap profiling
  - It’s a good idea to force garbage collection before dumping the heap profile to avoid false assumptions. This can be done via the web UI at http://localhost/debug/pprof/heap?gc=1.
  - Snapshots can be compared with go tool pprof -http=:8080 -diff_base <file2> <file1>.
- Goroutine profiling
  - We can inspect to check for “goroutine leaks”.
- Block profiling
  - Useful to check performance affected by blocking calls. Must call runtime.SetBlockProfileRate to enable it. This profiler stays on until the program ends.
- Mutex profiling
  - Useful to check performance affected by mutex locks. Must call runtime.SetMutexProfileFraction to enable it. This profiler stays on until the program ends.
- A single profiler should be enabled at a time.
Execution tracer
- Used to check how garbage collection performs, how goroutines are scheduled, and how the scheduler behaves.
- It can be enabled with go test -bench=. -v -trace=trace.out or by downloading a trace file with /debug/pprof/trace?debug=0.
- The granularity of the tracer is per goroutine (unlike CPU profiling, which is per function).

99 - Not understanding how the GC works

debug.FreeOSMemory() can be used to force the garbage collector to release memory to the OS.
GODEBUG=gctrace=1 can be used to see the garbage collector in action.
One can force a large allocation of memory with something like var min = make([]byte, 1_000_000_000) so that the garbage collector will only trigger once the heap reaches twice the size of the allocation, which can reduce the impact on latency if otherwise many garbage collections would be triggered in a short period of time. On most systems, this leads to a lazy allocation (in the virtual address space). This is useful when the heap peak is known in advance.

100 - Not understanding the impacts of running Go in Docker and Kubernetes

CPU throttling can occur in Docker and Kubernetes, leading to performance degradation.
GOMAXPROCS should be set to the number of CPUs available to the container.

Resources and references

Book summary: Dive into Design Patterns

Sébastien Lavoie — Sat, 09 Mar 2024 12:53:00 -0600

Introduction

In the vast expanse of software engineering, design patterns and principles stand as lighthouses, guiding developers through the complex process of crafting robust, scalable, and maintainable software. Among these patterns, the Object-Oriented Programming (OOP) principles, Software Design Principles, and various design patterns provide a framework that seasoned and novice developers alike can leverage to solve common software design challenges. This article constitutes a book summary of Dive into Design Patterns written by Alexander Shvets. Beautifully illustrated, it delves into the core aspects of OOP, explores essential software design principles, and unveils a catalog of design patterns categorized into Creational, Structural, and Behavioral patterns. Here, each section is crafted to mimic the same structure as the book to offer a concise yet (hopefully) comprehensive overview, equipped with examples and analogies to illuminate these concepts further.

A brief OOP overview

Class attributes are called fields.
Methods define the behavior of a class.
Fields and methods are the members of their class.
Objects are concrete instances of a class.
Class hierarchies
- Parent = superclass — inherit from parent.
Abstraction: models the behaviors of a real-world object.
Encapsulation
- Hide parts of state and behaviors, interfacing with a simpler abstraction (e.g., starting a car engine doesn’t require knowing anything about mechanics).
- Private fields and methods are hidden outside of a class.
- Protected fields and methods are also hidden outside of a class but accessible to subclasses.
Interfaces look like classes with methods only.
Inheritance (arrow with empty triangle head, going from subclass to parent class)
- Build subclasses on top of superclasses.
- Allows code reuse.
- Subclasses automatically share the same interface as their parent class.
- One class inherits interface and implementation of another and can extend it. It can be treated as the superclass and depends on it.
Polymorphism
- The real type of an object is not required to be known to use it (e.g., if a class inherits from Animal and implements an abstract method makeSound, then we can call makeSound on that class regardless of what type of animal this is).
Relations between objects
- Dependency (dashed lined with arrow)
- Code that depends on interfaces or abstract classes lead to weaker dependency between classes.
- One class can affect the code from another class.
- Association (simple arrow pointing to the object being used, bi-direction allowed)
- When an object uses or interacts with another.
- One class knows about the other and uses it.
- Aggregation (line with empty diamond at the container end, an arrow at the end pointing toward the component)
- One object “has” a collection of others and serves as a container (e.g., a department has many professors).
- One class is made of another one and depends on it.
- The component can exist without the container.
- Composition (drawn like an aggregation, filling in the diamond)
- Specific kind of aggregation, where the component can only exist as part of the container.
- Like aggregation, but the first class also manages the life cycle of the class it depends on.
- Implementation
- One class defines methods from an interface. The class can be treated as the interface and depends on it.

Introduction to Design Patterns

Design patterns are solutions to common problems.
A pattern is a general concept for solving a particular problem.
A pattern is more high-level than an algorithm.
Architectural patterns are high-level. They are used to design the architecture of an entire application and can usually be implemented in any programming language.
Creational patterns are a way to increase flexibility and code reuse with object creation mechanisms.
Structural patterns are a way of assembling objects/classes into larger structures, keeping structures flexible and efficient.
Behavioral patterns deal with effective communication and assignment of responsibilities between objects.

Software Design Principles

Features of good software design include:

Code reuse
Extensibility

Design Principles

Encapsulate what varies.
- Minimize the effect caused by changes.
Program to an interface, not an implementation.
- A design can be seen as flexible when it doesn’t break if it can be extended without breaking existing code.
Favor composition over inheritance.
- Problems:
  - A subclass must implement all abstract methods from the superclass even if they’re not used.
  - Behavior of overridden methods must remain compatible with the base class.
  - The superclass is not well encapsulated, because all subclasses are aware of its implementation details. Furthermore, if a subclass needs to be extended, the superclass might be modified to accommodate this need.
  - Changes in the superclass can break the subclasses, leading to tight coupling.
  - It can lead to the creation of many hierarchies of classes to combine them over multiple dimensions.
- Inheritance represents the “is a” relationship, while composition represents a “has a” relationship.

SOLID Principles

Single-responsibility principle: “a class should have just one reason to change”.
Open-closed principle: classes should be open for extension but closed for modification.
Liskov Substitution Principle: when extending a class, you should be able to pass objects of the subclass in place of objects of the parent class without breaking the client code.
- Parameter types in a method of a subclass should match or be more abstract than parameter types in the method of the superclass.
- The return type in a method of a subclass should match or be a subtype of the return type in the method of the superclass.
- A method in a subclass shouldn’t throw types of exceptions which the base method isn’t expected to throw.
- A subclass shouldn’t strengthen pre-conditions, e.g. if a parameter requires an int, the subclass shouldn’t restrict the int to be positive.
- A subclass shouldn’t weaken post-conditions, e.g. if a database connection is expected to be closed in a class, don’t leave it open in a subclass.
- Invariants of a superclass must be preserved. For this, it’s safer to add fields to a class rather than modify existing ones.
- A subclass shouldn’t change values of private fields of the superclass, this is possible e.g. in Python or JavaScript where private members aren’t even protected.
Interface Segregation Principle
- Clients shouldn’t be forced to depend on methods they do not use, trim the fat off of big interfaces.
Dependency Inversion Principle
- High-level classes shouldn’t depend on low-level classes. Both should depend on abstractions. Abstractions shouldn’t depend on details. Details should depend on abstractions.

Catalog of Design Patterns

Creational Design Patterns

These patterns deal with object creation mechanisms, trying to create objects in a manner suitable to the situation.

Factory Method (virtual constructor)

Analogy
- The hiring process (factory method) of a company (client) may differ for each department (concrete creator), which will hire employees (products of the concrete creator).
When to use it
- "[…] you don’t know beforehand the exact types and dependencies of the objects your code should work with".
- To provide users of your library or framework with a way to extend its internal components.
- To save system resources by reusing existing objects instead of rebuilding them each time.
Implementation
- Define an interface that all products will implement.
- Add a factory method to the creator class.
- In the creator’s code, replace direct calls to product constructors with calls to the factory method.
- The factory method might need a parameter to decide which product type to create.
- Initially, the factory method might have a switch or similar logic to decide which product to instantiate.
- Create subclasses of the creator for each product type, overriding the factory method to handle specific product creation.
- Either create more subclasses or use a parameter in the factory method to specify the product type if there are many product types.
Pros
- No tight coupling between the creator and the concrete products.
- Single Responsibility Principle: the creator is responsible for creating products, and the products are responsible for their own creation.
- Open/Closed Principle: you can introduce new product types without changing the existing code.
Cons
- The code might become more complicated than it should be.
Relation to other patterns
- Abstract Factory is a more complex version of Factory Method.
- Factory Method is a specialization of Template Method.
- Prototype doesn’t require subclassing, but it’s not always possible to use it with a Factory Method.

 go
 
 
 
package main

import "fmt"

// Vehicle is the product
type Vehicle interface {
 Brake() string
 Drive() string
}

// VehicleFactory is the abstract factory
type VehicleFactory interface {
 CreateVehicle(vehicleType string) Vehicle
}

// CheapVehicleFactory is a concrete factory producing cheap stuff
type CheapVehicleFactory struct{}

func (f *CheapVehicleFactory) CreateVehicle(vehicleType string) Vehicle {
 switch vehicleType {
 case "Car":
 return &CheapCar{}
 case "Bike":
 return &CheapBike{}
 default:
 return nil
 }
}

// NiceVehicleFactory is a concrete factory producing nice stuff
type NiceVehicleFactory struct{}

func (f *NiceVehicleFactory) CreateVehicle(vehicleType string) Vehicle {
 switch vehicleType {
 case "Car":
 return &NiceCar{}
 case "Bike":
 return &NiceBike{}
 default:
 return nil
 }
}

// CheapCar is a concrete product implementing the Vehicle interface that's not so reliable
type CheapCar struct{}

// NiceCar is a concrete product implementing the Vehicle interface that's more likely to work
type NiceCar struct{}

// CheapBike is what it is
type CheapBike struct{}

// NiceBike is nice
type NiceBike struct{}

func (c *CheapCar) Brake() string {
 return "Squeeeeeeeak..."
}

func (c *NiceCar) Brake() string {
 return "Shhh."
}

func (c *CheapCar) Drive() string {
 return "Vroom poof poof vroom poof poof vroom"
}

func (c *NiceCar) Drive() string {
 return "VROOOOOM"
}

func (b *CheapBike) Brake() string {
 return "iiiiiiiiik"
}

func (b *NiceBike) Brake() string {
 return "(silence)"
}

func (b *CheapBike) Drive() string {
 return "kwik kwik kwik"
}

func (b *NiceBike) Drive() string {
 return "(silence)"
}

func produceVehicles(factory VehicleFactory) {
 car := factory.CreateVehicle("Car")
 fmt.Println("Car:")
 fmt.Println("drive:", car.Drive())
 fmt.Println("stop: ", car.Brake())

 bike := factory.CreateVehicle("Bike")
 fmt.Println("Bike:")
 fmt.Println("ride: ", bike.Drive())
 fmt.Println("brake:", bike.Brake())
}

func main() {
 momsGarage := &CheapVehicleFactory{}
 uncleBobsMansion := &NiceVehicleFactory{}

 fmt.Println("Cheap Vehicles\n-----------------")
 produceVehicles(momsGarage)

 fmt.Println("\nNice Vehicles\n---------------")
 produceVehicles(uncleBobsMansion)

 // Output:
 /*
 Cheap Vehicles
 -----------------
 Car:
 drive: Vroom poof poof vroom poof poof vroom
 stop: Squeeeeeeeak...
 Bike:
 ride: kwik kwik kwik
 brake: iiiiiiiiik

 Nice Vehicles
 ---------------
 Car:
 drive: VROOOOOM
 stop: Shhh.
 Bike:
 ride: (silence)
 brake: (silence)
 */
}

Abstract Factory

Analogy
- A car manufacturing company has different (abstract) factories in different parts of the world to produce vehicle whose requirements are adapted to the local needs.
When to use it
- It can create families of related or dependent objects without the need to specify their concrete classes.
- It makes the system independent from how the objects are created.
- It can be used where the set of objects may change dynamically while keeping the same functionality.
Implementation
- Define the abstract factory interface.
- Create concrete factory classes.
- Define the product interfaces.
- Create concrete product classes.
- Use the factory.
Pros
- Isolation from concrete classes.
- Can swap product families.
- Makes products consistent.
Cons
- More complex than the Factory Method pattern.
- More difficult to extend, having the need to change many interfaces.
- Works best with sets of families that don’t change too much or too often.
Relation to other patterns
- It can use the Factory Method to create the products.
- Builder allows constructing more complex objects one step at a time.
- Prototype can be used when creating products by cloning prototypes.

 go
 
 
 
package main

import "fmt"

// Product interfaces
type Car interface {
 Drive() string
}
type Bicycle interface {
 Ride() string
}

// Concrete products
type MercedesCar struct{}
type BMWCar struct{}
type CannondaleBicycle struct{}
type SpecializedBicycle struct{}

func (t *MercedesCar) Drive() string {
 return "Driving Mercedes"
}
func (t *BMWCar) Drive() string {
 return "Driving BMW"
}
func (t *CannondaleBicycle) Ride() string {
 return "Riding Cannondale"
}
func (t *SpecializedBicycle) Ride() string {
 return "Riding Specialized"
}

// Abstract factory
type VehicleFactory interface {
 CreateCar() Car
 CreateBicycle() Bicycle
}

// Concrete factories
type MercedesFactory struct{}
type BMWFactory struct{}
type SpecializedFactory struct{}

func (t *MercedesFactory) CreateCar() Car {
 return &MercedesCar{}
}

// Let's assume Mercedes make Cannondale bicycles
func (t *MercedesFactory) CreateBicycle() Bicycle {
 return &CannondaleBicycle{}
}

func (t *BMWFactory) CreateCar() Car {
 return &BMWCar{}
}

// Let's assume BMW also makes Specialized bicycles
func (t *BMWFactory) CreateBicycle() Bicycle {
 return &SpecializedBicycle{}
}

// Let's assume Specialized makes its own bicycles
func (t *SpecializedFactory) CreateBicycle() Bicycle {
 return &SpecializedBicycle{}
}

func produceVehicles(factory VehicleFactory) {
 car := factory.CreateCar()
 fmt.Println(car.Drive())

 bicycle := factory.CreateBicycle()
 fmt.Println(bicycle.Ride())
}

func main() {
 mercedesFactory := &MercedesFactory{}
 bmwFactory := &BMWFactory{}

 fmt.Println("Mercedes vehicles:")
 produceVehicles(mercedesFactory)

 fmt.Println("\nBMW vehicles:")
 produceVehicles(bmwFactory)

 // This doesn't work because SpecializedFactory doesn't implement CreateCar()
 // produceVehicles(&SpecializedFactory{})

 // Output:
 /*
 Mercedes vehicles:
 Driving Mercedes
 Riding Cannondale

 BMW vehicles:
 Driving BMW
 Riding Specialized
 */
}

Builder

Analogy
- There are many ways to build a house, so it may be simpler and will be more flexible piecing the different requirements together rather than trying to find a pre-made house tailored to one’s needs.
When to use it
- When there’s a need for complex objects that can be built step by step (e.g., a text-to-speech [TTS] service may have dozens of options to set the pitch, speed, volume gain, etc. but many settings can be left at their default value).
- This pattern also works well when there are many optional components. In a TTS service, this could mean hooking up the result to post-process the audio, perform a TTS on multiple services at once, etc.
Implementation
- Define a Product class, which must have the common construction steps among all available products.
- There must be a Builder interface to define the methods used to construct the objects.
- Concrete builders are needed to implement the Builder interface in different ways (e.g., a wall builder for a wooden cabin vs. a brick house).
- There can be an optional Director class, which makes it easier to use to builders (e.g., for houses, there could be a MakeCabin method to call the appropriate builders in order to get the expected result, which is done in a single call to the Director instead of possibly chaining many invocations of a concrete builder).
- To use it, the client creates a Builder, passes the Builder to the Director to construct the product and the product is obtained from the Builder (e.g., Builder.getResult).
Pros
- It is flexible.
- It is reusable.
- It encapsulates complex construction logic inside the Builder.
Cons
- For simpler objects, it can make the code unnecessarily convoluted.
- Can be a bit painful to work with if many different products need to be created and used all at once.
Relation to other patterns
- Factory Method and Abstract Factory are simpler and create objects in a single step. The Builder is more complex but allows more control over the construction process.
- While Builder builds a complex object one step at a time, Prototype clones objects.

 go
 
 
 
package main

import "fmt"

// Candy is the Product
type Candy struct {
 Sugar int
 Flavor string
 Color string
 Packaging string
}

// CandyBuilder is the abstract Builder interface
type CandyBuilder interface {
 SetSugar(sugar int) CandyBuilder
 SetFlavor(flavor string) CandyBuilder
 SetColor(color string) CandyBuilder
 SetPackaging(packaging string) CandyBuilder
 Build() Candy
}

// SimpleCandyBuilder is a Concrete Builder
type SimpleCandyBuilder struct {
 candy Candy
}

func NewSimpleCandyBuilder() *SimpleCandyBuilder {
 return &SimpleCandyBuilder{candy: Candy{}}
}

func (b *SimpleCandyBuilder) SetSugar(sugar int) CandyBuilder {
 b.candy.Sugar = sugar
 return b
}

func (b *SimpleCandyBuilder) SetFlavor(flavor string) CandyBuilder {
 b.candy.Flavor = flavor
 return b
}

func (b *SimpleCandyBuilder) SetColor(color string) CandyBuilder {
 b.candy.Color = color
 return b
}

func (b *SimpleCandyBuilder) SetPackaging(packaging string) CandyBuilder {
 b.candy.Packaging = packaging
 return b
}

func (b *SimpleCandyBuilder) Build() Candy {
 return b.candy
}

type CandyDirector struct {
 builder CandyBuilder
}

func NewCandyDirector(builder CandyBuilder) *CandyDirector {
 return &CandyDirector{
 builder: builder,
 }
}

// ConstructPeppermintCandy creates a peppermint candy with green packaging.
func (d *CandyDirector) ConstructPeppermintCandy() Candy {
 return d.builder.SetSugar(10).
 SetFlavor("Peppermint").
 SetColor("White").
 SetPackaging("Green").
 Build()
}

// ConstructCaramelCandy creates a caramel candy with yellow packaging.
func (d *CandyDirector) ConstructCaramelCandy() Candy {
 return d.builder.SetSugar(15).
 SetFlavor("Caramel").
 SetColor("Brown").
 SetPackaging("Yellow").
 Build()
}

func main() {
 builder := NewSimpleCandyBuilder()
 director := NewCandyDirector(builder)

 peppermintCandy := director.ConstructPeppermintCandy()
 fmt.Printf("Peppermint Candy: %+v\n", peppermintCandy)

 caramelCandy := director.ConstructCaramelCandy()
 fmt.Printf("Caramel Candy: %+v\n", caramelCandy)

 // Output:
 // Peppermint Candy: {Sugar:10 Flavor:Peppermint Color:White Packaging:Green}
 // Caramel Candy: {Sugar:15 Flavor:Caramel Color:Brown Packaging:Yellow}
}

Prototype (clone)

Analogy
- Copying genetic information from one cell to another is very complex, while mitotic cell division is efficient and effective.
When to use it
- To create new objects by copying an existing object (the prototype).
- New instantiation of a complex object may be inconvenient.
Implementation
- Define an interface with a clone method.
- Implement the prototype interface in classes that need to be cloned.
- Use the clone method to create new objects.
Pros
- Cloning can be more efficient than creating new instances, especially if instantiation is resource-intensive.
- Cloning allows for dynamic creation and composition of instances at runtime, which brings greater flexibility.
- Prototypes can be managed and modified at runtime, which makes them dynamic.
Cons
- Deep cloning complex objects with intricate references can be challenging.
- It’s not always clear how cloning should behave for each object, especially in complex class hierarchies.
- Cloning can lead to unexpected issues, such as shared references, if not handled correctly.
Relation to other patterns
- Factory Method uses a single method to create new objects, Prototype does it through cloning.
- Abstract Factory and Builder can be combined with Prototype when complex objects are required to be built or cloned.
- Prototypes are often at odds with Singletons, as the Prototype pattern involves creating new object instances.

 go
 
 
 
package main

import "fmt"

// Animal is the prototype interface that all animals must implement
type Animal interface {
 Clone() Animal
 GetInfo() string
}

// Sheep is a concrete prototype that implements the Animal interface
type Sheep struct {
 Name string
 Age int
}

// Clone creates a new Sheep instance with the same data
func (s *Sheep) Clone() Animal {
 // Create a new Sheep instance with the same data
 return &Sheep{Name: s.Name, Age: s.Age}
}

// GetInfo returns the sheep's name and age
func (s *Sheep) GetInfo() string {
 return fmt.Sprintf("Sheep Name: %s, Age: %d", s.Name, s.Age)
}

// Dog is another concrete prototype that implements the Animal interface
type Dog struct {
 Name string
 Breed string
}

// Clone creates a new Dog instance with the same data
func (d *Dog) Clone() Animal {
 return &Dog{Name: d.Name, Breed: d.Breed}
}

// GetInfo returns the dog's name and breed
func (d *Dog) GetInfo() string {
 return fmt.Sprintf("Dog Name: %s, Breed: %s", d.Name, d.Breed)
}

type PrototypeRegistry struct {
 items map[string]Animal
}

func NewPrototypeRegistry() *PrototypeRegistry {
 return &PrototypeRegistry{items: make(map[string]Animal)}
}

func (r *PrototypeRegistry) Register(name string, animal Animal) {
 r.items[name] = animal
}

func (r *PrototypeRegistry) Clone(name string) (Animal, error) {
 if animal, exists := r.items[name]; exists {
 return animal.Clone(), nil
 }
 return nil, fmt.Errorf("prototype not found: %s", name)
}

func main() {
 registry := NewPrototypeRegistry()
 registry.Register("sheep", &Sheep{Name: "Dolly", Age: 4})
 registry.Register("dog", &Dog{Name: "Fido", Breed: "Bulldog"})

 clonedAnimal, err := registry.Clone("sheep")
 if err != nil {
 fmt.Println("Error cloning sheep:", err)
 return
 }

 clonedSheep, ok := clonedAnimal.(*Sheep)
 if !ok {
 fmt.Println("Type assertion failed: not a *Sheep")
 return
 }
 clonedSheep.Name = "Molly"
 fmt.Println(clonedSheep.GetInfo())

 clonedAnimal2, err := registry.Clone("dog")
 if err != nil {
 fmt.Println("Error cloning dog:", err)
 return
 }

 clonedDog, ok := clonedAnimal2.(*Dog)
 if !ok {
 fmt.Println("Type assertion failed: not a *Dog")
 return
 }
 clonedDog.Name = "Rex"
 fmt.Println(clonedDog.GetInfo())

 // Output:
 /*
 Sheep Name: Molly, Age: 4
 Dog Name: Rex, Breed: Bulldog
 */
}

Singleton

Analogy
- The government of a country is (ideally…) a singleton. There is a single group recognized to act as the Government class.
When to use it
- It does two things (not a single responsibility…):
  - It ensures that a class has only one instance;
  - It provides a global point of access to that instance.
- It is often used to manage shared resources, such as a database connection.
Implementation
- A private variable in the package holds the instance.
- Ensure safe access in concurrent environments (e.g., using sync.Once).
- A public function provides global access to the instance.
Pros
- Provides controlled access to a unique instance.
- Instance is created only when needed (lazy Initialization).
- Easily accessible throughout the application (global access).
Cons
- It can introduce a global state in the application, which is generally discouraged in software design.
- Singleton can make unit testing tricky due to its global state.
- It needs careful handling in a concurrent environment to avoid multiple instantiations.
Relation to other patterns
- Singleton can be combined with Factory Method. The factory method ensures that a single instance is created and returned.
- Abstract Factory, Builder, Prototype: these patterns are about creating complex objects, which is a different focus from Singleton’s objective of a single instance.

 go
 
 
 
package main

import (
 "fmt"
 "sync"
)

type ConfigurationData struct {
 DatabaseURL string
}

// ConfigurationManager is a singleton that holds the configuration for the application.
type ConfigurationManager struct {
 config *ConfigurationData
}

var (
 instance *ConfigurationManager
 once sync.Once
)

// GetInstance returns the singleton instance of the ConfigurationManager.
func GetInstance() *ConfigurationManager {
 once.Do(func() {
 instance = &ConfigurationManager{
 config: &ConfigurationData{
 DatabaseURL: "localhost:4321", // Default value
 },
 }
 })
 return instance
}

// UpdateConfig updates configuration if needed
func (m *ConfigurationManager) UpdateConfig(newConfig *ConfigurationData) {
 m.config = newConfig
}

func main() {
 config1 := GetInstance()
 config2 := GetInstance()

 fmt.Println(config1 == config2) // true
}

Structural Design Patterns

These patterns focus on ways to compose objects to form larger structures efficiently and flexibly.

Adapter

Analogy
- An adapter may be needed when traveling abroad to charge devices. While it doesn’t change the device’s functionality, it allows it to work in situations where it would normally be incompatible.
When to use it
- To allow two incompatible interfaces to work together.
- Use an existing class that has an incompatible interface with the rest of the code.
Implementation
- Implement a client interface.
- An adaptee is a class with an incompatible interface (the one that gets adapted).
- An adapter is a class that implements the client interface, translating calls to the adaptee’s interface (the one that adapts the incompatible code).
Pros
- Allows otherwise incompatible interfaces to work together.
- Enables the reuse of existing classes without altering their code.
- Introduces only a small amount of extra code to bridge the compatibility gap.
Cons
- Can add complexity to the code, especially if there are many adaptions.
- The adapter pattern involves an extra layer of abstraction, which can complicate the code and increase overhead.
Relation to other patterns
- While Adapter is used to change an interface, Decorator is used to add responsibilities to objects without modifying their interface.
- The Bridge pattern separates an object’s interface from its implementation so the two can vary independently. Adapter is meant to change the interface of an existing object.
- A Façade provides a simplified interface to a complex subsystem, not necessarily designed for incompatible interfaces, unlike Adapter.

 go
 
 
 
package main

import "fmt"

type Outlet interface {
 InsertInOutlet() string
}

type Pluggable interface {
 InsertInUSOutlet() string
}

type USPlug struct{}

type USPlugAdapter struct {
 usPlug Pluggable
}

func (plug *USPlug) InsertInUSOutlet() string {
 return "Inserted in US Outlet"
}

func NewUSPlugAdapter(usPlug Pluggable) *USPlugAdapter {
 return &USPlugAdapter{usPlug: usPlug}
}

func (adapter *USPlugAdapter) InsertInOutlet() string {
 return adapter.usPlug.InsertInUSOutlet() + ", adapted to fit an Outlet"
}

func ChargeDevice(outlet Outlet) string {
 return outlet.InsertInOutlet()
}

func main() {
 usPlug := &USPlug{}
 adaptedPlug := NewUSPlugAdapter(usPlug)

 result := ChargeDevice(adaptedPlug)
 fmt.Println(result) // Inserted in US Outlet, adapted to fit an Outlet
}

Bridge

Analogy
- A universal remote control bridges the functionality gap between different devices (e.g., it can turn up or down the volume on all devices) with a common interface.
When to use it
- Use it to separate the abstraction (high-level logic) from the implementation (low-level logic), allowing them to be developed independently.
- When you want to avoid a permanent binding between the abstraction and its implementation, particularly when they both may vary frequently.
Implementation
- The abstraction interface/class is a higher-level control layer that delegates the work to the implementation layer.
- The implementor interface defines the interface for the implementation classes.
- Concrete implementations are classes that provide specific implementations of the implementor interface.
- An optional refined abstraction can be used to extend the abstraction to provide more control.
Pros
- Abstractions and implementations can be extended independently.
- Separates high-level and low-level logic into different classes.
- The abstraction and its implementation can be developed and maintained separately.
Cons
- Can increase complexity due to the introduction of additional layers.
- More effort is needed to set up the structure, which might not be necessary for simpler cases.
Relation to other patterns
- The Bridge pattern is often compared to the Adapter pattern. Bridge is designed to separate an interface from its implementation, while Adapter is meant to make unrelated classes work together.
- Bridge is often used in a similar way to Strategy. Strategy typically encapsulates algorithms: Bridge separates an interface from its implementation.
- Abstract Factory can work with Bridge as it can create instances of different classes that implement the abstract interface.
- Builder can be combined with Bridge: builders are the implementations and the director is the abstraction.

 go
 
 
 
package main

import "fmt"

// The Device interface (the abstraction) defines the operations that all
// concrete device implementations must support.
type Device interface {
 TurnOn()
 TurnOff()
 IsEnabled() bool
 SetVolume(volume int)
 GetVolume() int
}

// TV and Radio are concrete implementations of the Device interface.
type TV struct {
 isEnabled bool
}

func (t *TV) TurnOn() {
 t.isEnabled = true
}

func (t *TV) TurnOff() {
 t.isEnabled = false
}

func (t *TV) IsEnabled() bool {
 return t.isEnabled
}

// SetVolume for TV
func (t *TV) SetVolume(volume int) {
 fmt.Printf("TV volume set to %d\n", volume)
}

// GetVolume for TV
func (t *TV) GetVolume() int {
 return 50 // Placeholder
}

type Radio struct {
 isEnabled bool
}

func (r *Radio) TurnOn() {
 r.isEnabled = true
}

func (r *Radio) TurnOff() {
 r.isEnabled = false
}

func (r *Radio) IsEnabled() bool {
 return r.isEnabled
}

// SetVolume for Radio
func (r *Radio) SetVolume(volume int) {
 fmt.Printf("🔉 Radio volume set to %d\n", volume)
}

// GetVolume for Radio
func (r *Radio) GetVolume() int {
 return 30 // Placeholder
}

type RemoteControl struct {
 device Device
 name string
}

func NewRemoteControl(device Device, name string) *RemoteControl {
 fmt.Println("NewRemoteControl", name)
 return &RemoteControl{device: device, name: name}
}

func (r *RemoteControl) TogglePower() {
 if r.device.IsEnabled() {
 fmt.Println(r.name, "🔴 turning off")
 r.device.TurnOff()
 } else {
 fmt.Println(r.name, "🟢 turning on")
 r.device.TurnOn()
 }
}

type AdvancedRemoteControl struct {
 *RemoteControl // Embedding RemoteControl to inherit its methods
}

func NewAdvancedRemoteControl(device Device, name string) *AdvancedRemoteControl {
 return &AdvancedRemoteControl{RemoteControl: NewRemoteControl(device, name)}
}

func (a *AdvancedRemoteControl) SetDeviceVolume(volume int) {
 a.device.SetVolume(volume)
}

func NewTV() *TV {
 return &TV{isEnabled: false}
}

func NewRadio() *Radio {
 return &Radio{isEnabled: false}
}

func main() {
 // TV without advanced remote
 tv := NewTV()
 remote := NewRemoteControl(tv, "Samsung TV")
 remote.TogglePower() // Turns TV on
 remote.TogglePower() // Turns TV off

 // Radio with advanced remote
 radio := NewRadio()
 radioRemote := NewAdvancedRemoteControl(radio, "Sony Radio")
 radioRemote.TogglePower()
 radioRemote.SetDeviceVolume(30)
 radioRemote.TogglePower()

 // Output:
 /*
 NewRemoteControl Samsung TV
 Samsung TV turning on
 Samsung TV turning off
 NewRemoteControl Sony Radio
 Sony Radio turning on
 Radio volume set to 30
 Sony Radio set volume to 30
 Sony Radio turning off
 */
}

Composite (object tree)

Analogy
- The military is a tree structure where orders come from the top, are passed down the divisions, brigades and so on where the leaves are the actual soldiers.
When to use it
- When you want to work with a tree structure and have to perform operations on both individual elements and groups of elements in the same way.
Implementation
- There is a component interface for individual objects and their compositions which defines operations that can be performed on both.
- A leaf represents individual objects in the structure.
- A composite represents a composite of objects (a group of Leaf objects) and implements the component interface.
Pros
- Makes it easier to work with complex tree structures by allowing you to treat individual objects and compositions uniformly.
- Can add and remove new types of components without changing existing code.
- Client code can treat composite structures and individual objects uniformly.
Cons
- Components that do not have much in common end up being overgeneralized.
- The implementation can become overly complex if the hierarchy gets too deep or if extensive functionality is required.
Relation to other patterns
- Builder can be used with Composite because its construction steps can be recursive.
- Chain of Responsibility passes a request along a chain of objects until one of them handles it, Composite arranges objects in a tree structure and works with these structures collectively.
- Iterators can be used to traverse a Composite tree.
- Visitor can be used to execute an operation on a whole tree.
- Shared leaves from the Composite tree can be implemented with Flyweights to reduce RAM usage.
- Decorator only has one component and augments it by wrapping it with new functionality while Composite just runs the logic on all child components.

 go
 
 
 
package main

import (
 "fmt"
 "strings"
)

// Command is the component interface that all commands must implement.
type Command interface {
 Execute()
 Undo()
}

type Document struct {
 content string
}

// WriteCommand is a leaf that implements the Command component interface.
type WriteCommand struct {
 document *Document
 addition string
}

// SaveCommand is another leaf.
type SaveCommand struct {
 document *Document
}

func (w *WriteCommand) Execute() {
 w.document.content += w.addition
 fmt.Printf("Document after writing: %s\n", w.document.content)
}

func (w *WriteCommand) Undo() {
 fmt.Printf("Undo writing: %s\n", w.addition)
 w.document.content = strings.TrimSuffix(w.document.content, w.addition)
}

func (s *SaveCommand) Execute() {
 fmt.Printf("Document saved: %s\n", s.document.content)
}

func (s *SaveCommand) Undo() {
 fmt.Println("Restore to the last saved point...")
}

type CommandManager struct {
 history []Command
}

func (m *CommandManager) ExecuteCommand(cmd Command) {
 cmd.Execute()
 m.history = append(m.history, cmd)
}

func (m *CommandManager) Undo() {
 if len(m.history) == 0 {
 return
 }
 cmd := m.history[len(m.history)-1]
 cmd.Undo()
 m.history = m.history[:len(m.history)-1]
}

// CompositeCommand is a composite that implements the Command component interface.
type CompositeCommand struct {
 commands []Command
}

func (c *CompositeCommand) Execute() {
 for _, cmd := range c.commands {
 cmd.Execute()
 }
}

func (c *CompositeCommand) Undo() {
 for i := len(c.commands) - 1; i >= 0; i-- {
 c.commands[i].Undo()
 }
}

func (c *CompositeCommand) AddCommand(cmd Command) {
 c.commands = append(c.commands, cmd)
}

func main() {
 doc := &Document{}
 write1 := &WriteCommand{document: doc, addition: "Hello "}
 write2 := &WriteCommand{document: doc, addition: "World"}
 save := &SaveCommand{document: doc}

 compositeCommand := &CompositeCommand{}
 compositeCommand.AddCommand(write1)
 compositeCommand.AddCommand(write2)
 compositeCommand.AddCommand(save)

 commandManager := &CommandManager{}
 commandManager.ExecuteCommand(compositeCommand)

 fmt.Println("\nUndoing Composite Command:")
 commandManager.Undo()

 fmt.Printf("\nFinal Document Content: %s\n", doc.content)

 // Output:
 /*
 Document after writing: Hello
 Document after writing: Hello World
 Document saved: Hello World

 Undoing Composite Command:
 Restore to the last saved point...
 Undo writing: World
 Undo writing: Hello

 Final Document Content:
 */
}

Decorator (wrapper)

Analogy
- Wearing layers of clothes: without clothes, it’s cold; with a jacket, it’s warmer; with a raincoat on top, it’s warmer and dryer. The same person is “decorated” to be comfortable under different circumstances.
When to use it
- Used to add new functionality to an object dynamically, without altering its structure.
- Allows for extending an object’s behavior by wrapping it with additional functionality.
Implementation
- The Component interface defines the interface for objects that can have responsibilities added to them dynamically (e.g., a file with read() and write() methods).
- The Concrete component is an object to which additional responsibilities can be attached (e.g., a file with write() could first be compressed or encrypted).
- Decorator classes are wrappers that add responsibilities to the component. They implement the same interface as the component they decorate (e.g., taking the File object, there could be EncryptionDecorator and CompressionDecorator).
Pros
- Provides a flexible alternative to subclassing for extending functionality.
- Responsibilities can be added and removed at runtime.
- Allows for the combination of behaviors by stacking decorators on top of each other.
Cons
- Can lead to complex code structures and debugging difficulties, especially with multiple layers of decorators (order matters!).
- Creating an object decorated with multiple layers can require more complex setup code.
Relation to other patterns
- Unlike Adapter, which changes the behavior of an incompatible class to work with another class, Decorator doesn’t modify the original object and stacks functionality on top of it. Decorator can work recursively, while an Adapter can’t.
- Adapter leads to a wrapped object with a different interface. Proxy maintains the same interface. Decorator enhances it.
- Decorator is similar to Chain of Responsibility. But Decorator maintains consistency throughout the request, while a chain can stop at any point and handle multiple independent operations at once.
- A Decorator is like a Composite but with a single component. Composite also doesn’t add functionality. Prototype can be useful with these two patterns.
- Decorator lets one change the surface level of an object. Strategy lets one change to an entirely different object.

 go
 
 
 
package main

import "fmt"

// Component is the common interface for both the concrete component and the
// decorators. It can have multiple layers of decorators.
type Component interface {
 Operation() string
}

// ConcreteComponent is a concrete component that implements the Component
// interface, defining all the operations that can be altered by decorators.
type ConcreteComponent struct{}

func (c *ConcreteComponent) Operation() string {
 return "ConcreteComponent"
}

// Decorator is a base decorator class. It implements the Component interface
// and has a field for storing a reference to a wrapped object. The default
// implementation of the wrapped object does the actual work, while the
// decorator merely delegates to the wrapped object.
type Decorator struct {
 component Component
}

// Operation Decorator delegates all work to the wrapped component.
func (d *Decorator) Operation() string {
 return d.component.Operation()
}

// ConcreteDecoratorA calls the wrapped object and alter its result in some way.
type ConcreteDecoratorA struct {
 Decorator
}

// Operation Decorators may call parent implementation of the operation, instead of
// calling the wrapped object directly, simplifying extension of decorator classes.
func (d *ConcreteDecoratorA) Operation() string {
 return "ConcreteDecoratorA(" + d.Decorator.Operation() + " + AddedBehaviorA)"
}

func NewConcreteDecoratorA(component Component) *ConcreteDecoratorA {
 return &ConcreteDecoratorA{Decorator{component: component}}
}

type ConcreteDecoratorB struct {
 Decorator
}

func (d *ConcreteDecoratorB) Operation() string {
 pre := "PreBehaviorB | "
 base := d.Decorator.Operation() // Delegate to base Operation
 post := " | PostBehaviorB"
 return pre + "ConcreteDecoratorB(" + base + ")" + post
}

func NewConcreteDecoratorB(component Component) *ConcreteDecoratorB {
 return &ConcreteDecoratorB{Decorator{component: component}}
}

func main() {
 var decoratedComponent Component = &ConcreteComponent{}

 decoratedComponent = NewConcreteDecoratorA(decoratedComponent)
 decoratedComponent = NewConcreteDecoratorB(decoratedComponent)

 fmt.Println(decoratedComponent.Operation())
 // Output:
 // PreBehaviorB | ConcreteDecoratorB(ConcreteDecoratorA(ConcreteComponent + AddedBehaviorA)) | PostBehaviorB
}

Facade

Analogy
- Placing an order by phone is easy (call and get it delivered), but a bunch of operations must happen in the background (warehouse, packaging, suppliers, etc.), which the client has no direct knowledge of.
When to use it
- To provide a simplified interface to a complex subsystem. It doesn’t encapsulate the subsystem but offers an easier or more coherent interface for clients to interact with.
- Useful when dealing with a large body of code, such as a library or a framework.
Implementation
- A Facade interface defines an interface that abstracts the complex functionalities of the subsystem.
- Subsystem classes are the complex parts of the subsystem that the facade will simplify.
- A Facade class implements the facade interface and interacts with the subsystem classes on behalf of the client.
Pros
- Simplifies the interface for complex subsystems, making them easier to use.
- Clients are isolated from the complexities of the subsystem, reducing dependencies and making the system easier to use and maintain.
- Clients can work with a single unified interface instead of a set of complex and potentially confusing interfaces.
Cons
- The facade might limit access to certain features of the subsystem that advanced users might need.
- If not designed carefully, the facade can end up doing too much, becoming overly complex (a “God Object”).
Relation to other patterns
- The Adapter pattern changes the interface of an existing object; the Facade provides a simplified interface to a complex subsystem. The Facade doesn’t wrap the subsystem to change its interface but provides a simpler one.
- The Facade simplifies access to a complex system; the Decorator adds behavior to objects. They are both used to simplify client interactions but achieve this in different ways.
- Often, a Facade is implemented as a Singleton because usually one facade object is enough to manage the underlying systems.
- Abstract Factory can be an alternative when the main goal is to hide the way the subsystem objects are created in the client code.
- Facade makes a single object to represent a whole system while Flyweight makes lots of little objects.

 go
 
 
 
package main

// Example of a facade pattern

import (
 "fmt"
 "strings"
)

// TV is subsystem 1
type TV struct {
 isOn bool
}

func (t *TV) TurnOn() (string, error) {
 if t.isOn {
 return "", fmt.Errorf("❌ TV is already on")
 }
 t.isOn = true
 return "TV is on", nil
}

func (t *TV) TurnOff() (string, error) {
 if !t.isOn {
 return "", fmt.Errorf("❌ TV is already off")
 }
 t.isOn = false
 return "TV is off", nil
}

// SoundSystem is subsystem 2
type SoundSystem struct {
 isOn bool
}

func (s *SoundSystem) TurnOn() (string, error) {
 if s.isOn {
 return "", fmt.Errorf("❌ SoundSystem is already on")
 }
 s.isOn = true
 return "SoundSystem is on", nil
}

func (s *SoundSystem) TurnOff() (string, error) {
 if !s.isOn {
 return "", fmt.Errorf("❌ SoundSystem is already off")
 }
 s.isOn = false
 return "SoundSystem is off", nil
}

// DVDPlayer is subsystem 3
type DVDPlayer struct {
 isOn bool
}

func (d *DVDPlayer) Play() (string, error) {
 if d.isOn {
 return "", fmt.Errorf("❌ DVD is already playing")
 }
 d.isOn = true
 return "DVD is playing", nil
}

func (d *DVDPlayer) Stop() (string, error) {
 if !d.isOn {
 return "", fmt.Errorf("❌ DVD is already stopped")
 }
 d.isOn = false
 return "DVD stopped", nil
}

// HomeTheaterFacade provides a simplified interface to a complex subsystem
type HomeTheaterFacade struct {
 tv *TV
 soundSystem *SoundSystem
 dvdPlayer *DVDPlayer
}

// NewHomeTheaterFacade is a constructor for HomeTheaterFacade
func NewHomeTheaterFacade(tv *TV, soundSystem *SoundSystem, dvdPlayer *DVDPlayer) *HomeTheaterFacade {
 return &HomeTheaterFacade{
 tv: tv,
 soundSystem: soundSystem,
 dvdPlayer: dvdPlayer,
 }
}

// WatchMovie is a facade method
func (h *HomeTheaterFacade) WatchMovie() (string, error) {
 if h.tv == nil || h.soundSystem == nil || h.dvdPlayer == nil {
 return "", fmt.Errorf("❌ home theater is not properly set up")
 }

 response := NewOperationResponse()
 tvResponse, tvErr := h.tv.TurnOn()
 response.AddResponse(tvResponse, tvErr)
 soundSystemResponse, soundSystemErr := h.soundSystem.TurnOn()
 response.AddResponse(soundSystemResponse, soundSystemErr)
 dvdResponse, dvdErr := h.dvdPlayer.Play()
 response.AddResponse(dvdResponse, dvdErr)
 return response.String(), nil
}

// StopMovie is a facade method
func (h *HomeTheaterFacade) StopMovie() string {
 response := NewOperationResponse()
 dvdResponse, dvdErr := h.dvdPlayer.Stop()
 response.AddResponse(dvdResponse, dvdErr)
 soundSystemResponse, soundSystemErr := h.soundSystem.TurnOff()
 response.AddResponse(soundSystemResponse, soundSystemErr)
 tvResponse, tvErr := h.tv.TurnOff()
 response.AddResponse(tvResponse, tvErr)
 return response.String()
}

type OperationResponse struct {
 Responses []string
 Errors []error
}

func NewOperationResponse() *OperationResponse {
 return &OperationResponse{}
}

func (o *OperationResponse) AddResponse(response string, err error) {
 if response != "" {
 o.Responses = append(o.Responses, response)
 }
 if err != nil {
 o.Errors = append(o.Errors, err)
 }
}

func (o *OperationResponse) String() string {
 // Combine all responses and errors into a single string, don't display nil errors
 var responseStrings []string
 responseStrings = append(responseStrings, o.Responses...)
 return strings.Join(responseStrings, "\n")
}

func mapErrorsToStrings(errors []error) []string {
 var errorStrings []string
 for _, err := range errors {
 errorStrings = append(errorStrings, err.Error())
 }
 return errorStrings
}

func main() {
 tv := &TV{}
 soundSystem := &SoundSystem{}
 dvdPlayer := &DVDPlayer{}

 faultyHomeTheater := NewHomeTheaterFacade(tv, soundSystem, nil)
 fmt.Println("Trying to watch a movie with a faulty home theater...")
 faultyMovie, err := faultyHomeTheater.WatchMovie()
 if err != nil {
 fmt.Println(err)
 } else {
 fmt.Println(faultyMovie)
 }

 homeTheater := NewHomeTheaterFacade(tv, soundSystem, dvdPlayer)
 fmt.Println("About to watch a movie with a properly set up home theater...")
 movie, err := homeTheater.WatchMovie()
 if err != nil {
 return
 }
 fmt.Println(movie)

 fmt.Println("Someone tries to turn on the TV while it's already on...")
 _, err = tv.TurnOn()
 if err != nil {
 fmt.Println(err)
 }

 fmt.Println("Trying to start the movie again while it's already playing...")
 watchMovie, err := homeTheater.WatchMovie()
 if err != nil {
 return
 }
 fmt.Println(watchMovie)
 fmt.Println("Stopping the movie...")
 homeTheater.StopMovie()
 fmt.Println("Trying to stop the movie again while it's already stopped...")
 homeTheater.StopMovie()

 // Output:
 /*
 Trying to watch a movie with a faulty home theater...
 ❌ home theater is not properly set up
 About to watch a movie with a properly set up home theater...
 TV is on
 SoundSystem is on
 DVD is playing
 Someone tries to turn on the TV while it's already on...
 ❌ TV is already on
 Trying to start the movie again while it's already playing...
 Stopping the movie...
 Trying to stop the movie again while it's already stopped...
 */
}

Flyweight (cache)

Analogy
- A library (flyweight factory) lends books (flyweight objects with intrinsic state) to library members (context in which extrinsic state exists) without creating a resource nightmare (resources are shared as much as possible).
When to use it
- There’s a need to spawn a huge number of similar objects sharing some context.
- The intrinsic state can be extracted across different objects. E.g., actual trees of the same type may share attributes like texture and color (intrinsic state) but will have distinct attributes like their (x, y, z) position (extrinsic state).
Implementation
- Separate intrinsic and extrinsic states.
  - Get the fields that won’t change (intrinsic state). These are unchanging and unique data points. They are implemented via the Flyweight interface.
  - The extrinsic state is the contextual data that is unique to each object and cannot be shared.
- A concrete flyweight implements the Flyweight interface and stores intrinsic state that can be shared. The extrinsic state is passed as arguments to methods. Fields related to the intrinsic state must be immutable.
- A flyweight factory class can be created to avoid re-creating existing objects with a set comprising the exact same attributes. The intrinsic state is passed to the factory.
Pros
- Can save a lot of memory depending on context. This is only relevant when a very large number of objects are needed.
Cons
- While RAM may decrease, there can be a trade off with CPU as contextual data may need to be recalculated.
- This increases complexity substantially.
Relation to other patterns
- Shared leaf nodes of a Composite tree may be implemented as Flyweights.
- Flyweight creates lots of smaller objects, Facade creates a single large object representing the whole system.

 go
 
 
 
package main

import (
 "fmt"
 "strings"
)

// TextStyle is the flyweight interface
type TextStyle interface {
 ApplyStyle(text string) string
}

// BoldStyle is a concrete flyweight
type BoldStyle struct{}

func (b *BoldStyle) ApplyStyle(text string) string {
 return "**" + text + "**"
}

// UrgentStyle is a concrete flyweight
type UrgentStyle struct{}

func (u *UrgentStyle) ApplyStyle(text string) string {
 return "‼️ " + text + " ‼️"
}

// ItalicStyle is a concrete flyweight
type ItalicStyle struct{}

func (i *ItalicStyle) ApplyStyle(text string) string {
 return "_" + text + "_"
}

// BulletStyle is a concrete flyweight
type BulletStyle struct{}

func (b *BulletStyle) ApplyStyle(text string) string {
 return "* " + text
}

// StyleFactory is the flyweight factory
type StyleFactory struct {
 styles map[string]TextStyle
 compositeStyles map[string]*CompositeStyle // cache for composite styles
}

// NewStyleFactory is the flyweight factory constructor
func NewStyleFactory() *StyleFactory {
 return &StyleFactory{
 styles: make(map[string]TextStyle),
 compositeStyles: make(map[string]*CompositeStyle),
 }
}

func (f *StyleFactory) GetStyle(styleName string) (TextStyle, error) {
 if style, exists := f.styles[styleName]; exists {
 return style, nil
 }

 var style TextStyle
 switch styleName {
 case "bold":
 style = &BoldStyle{}
 case "italic":
 style = &ItalicStyle{}
 case "bullet":
 style = &BulletStyle{}
 default:
 return nil, fmt.Errorf("unknown style: %s", styleName)
 }

 f.styles[styleName] = style
 return style, nil
}

// Extend with the Composite pattern to get multiple styles applied to a text

// CompositeStyle is the composite class
type CompositeStyle struct {
 styles []TextStyle
}

// NewCompositeStyle is the composite constructor
func NewCompositeStyle(styles ...TextStyle) *CompositeStyle {
 return &CompositeStyle{styles: styles}
}

// ApplyStyle is the composite method
func (c *CompositeStyle) ApplyStyle(text string) string {
 for _, style := range c.styles {
 text = style.ApplyStyle(text)
 }
 return text
}

// AddStyle is the composite method
func (c *CompositeStyle) AddStyle(style TextStyle) {
 c.styles = append(c.styles, style)
}

func (f *StyleFactory) GetCompositeStyle(styleNames ...string) TextStyle {
 key := strings.Join(styleNames, "+") // Create a unique key for the combination

 if composite, exists := f.compositeStyles[key]; exists {
 return composite
 }

 var composite = NewCompositeStyle()
 for _, name := range styleNames {
 style, _ := f.GetStyle(name) // Handle error as appropriate
 composite.AddStyle(style)
 }

 f.compositeStyles[key] = composite
 return composite
}

type ConditionalStyle struct {
 condition func(string) bool
 style TextStyle
}

func NewConditionalStyle(condition func(string) bool, style TextStyle) *ConditionalStyle {
 return &ConditionalStyle{condition: condition, style: style}
}

func (c *ConditionalStyle) ApplyStyle(text string) string {
 if c.condition(text) {
 return c.style.ApplyStyle(text)
 }
 return text
}

func main() {
 factory := NewStyleFactory()

 boldStyle, _ := factory.GetStyle("bold")
 fmt.Println(boldStyle.ApplyStyle("Hello"))

 italicStyle, _ := factory.GetStyle("italic")
 fmt.Println(italicStyle.ApplyStyle("Go!"))

 // Reusing flyweights
 anotherBold, _ := factory.GetStyle("bold")
 fmt.Println(anotherBold.ApplyStyle("Flyweight bold"))

 bulletStyle, _ := factory.GetStyle("bullet")
 fmt.Println(bulletStyle.ApplyStyle("Flyweight list"))

 // Composite styles
 italicBulletStyle := factory.GetCompositeStyle("italic", "bullet")
 fmt.Println(italicBulletStyle.ApplyStyle("Italic bullet"))
 bulletItalicStyle := factory.GetCompositeStyle("bullet", "italic")
 fmt.Println(bulletItalicStyle.ApplyStyle("Bullet italic that's incorrect..."))

 // Apply bold style only if the text contains "important"
 importantCondition := func(text string) bool {
 return strings.Contains(text, "important")
 }
 importantBoldStyle := NewConditionalStyle(importantCondition, &BoldStyle{})

 // Apply urgent style to text that contains "urgent"
 urgentCondition := func(text string) bool {
 return strings.Contains(text, "urgent")
 }
 urgentStyle := NewConditionalStyle(urgentCondition, &UrgentStyle{})

 fmt.Println(importantBoldStyle.ApplyStyle("This is important and should be bold"))
 fmt.Println(urgentStyle.ApplyStyle("This is important but not bold"))
 fmt.Println(urgentStyle.ApplyStyle("This is urgent"))
 fmt.Println(importantBoldStyle.ApplyStyle("This is urgent but not emphasized"))

 // Output:
 /*
 **Hello**
 _Go!_
 **Flyweight bold**
 * Flyweight list
 * _Italic bullet_
 _* Bullet italic that's incorrect..._
 **This is important and should be bold**
 This is important but not bold
 ‼️ This is urgent ‼️
 This is urgent but not emphasized
 */
}

Proxy

Analogy
- Cash and credit cards are two ways of handling payment.
When to use it
- For lazy initialization (virtual proxy). A heavy service can be instantiated as needed instead of leaving it running.
- Similarly, for smart referencing (smart reference proxy). This means that if a heavy service needs to be up and running all the time but no clients are using it, it could be shut down until a client connects.
- For access control (protection proxy). For instance, a library implements the Spotify API but needs to limit access to non-paying users. A proxy interface can be used and internal logic can determine which customers can access the API at a given time. The Spotify API remains the same, but custom logic can run before and after its use.
- For logging requests. Instead of just calling an API, the proxy can implement logging and then call the API.
- For caching purposes. Instead of downloading a video over and over again via an API, the result can be cached.
Implementation
- Create a service Proxy interface that is used by the client (Subject interface). The proxy implements the same interface as the service (RealSubject) and can do stuff on top of it (e.g., data caching or logging).
- In most cases, the proxy object does some work and delegates the actual use of the service to the service itself from a field that references that service. The proxy class is instantiated with a reference to the service.
- Optionally, a factory method can be used.
- Also in option, lazy initialization can be implemented for the service object.
Pros
- The proxy transparently controls all aspects of the service and is indistinguishable from the service from the client’s perspective.
- The lifecycle of the service can be managed by the proxy class when appropriate, e.g. to reduce system resources.
- The proxy works even when the actual service could be down.
- New proxies can be added without changing existing code.
Cons
- Complexity increases as new proxies are added.
- It may cause delay in the response from the service if the proxy performs heavy operations.
Relation to other patterns
- Adapter provides a different interface, Proxy provides the same interface and Decorator provides an enhanced version of the interface.
- Proxy is indistinguishable from the original service, whereas Facade may be a simplified interface.
- Decorator is controlled by the client, while Proxy manages the whole lifecycle of the service.

 go
 
 
 
package main

import "fmt"

// RealDocument is the real object that the proxy will represent
type RealDocument struct {
 content string
}

type Displayable interface {
 Display() string
}

type Editable interface {
 Edit(newContent string)
}

// Display is the method that the proxy and the real object both implement, implementing the Displayable interface
func (d *RealDocument) Display() string {
 return "Displaying Document: " + d.content
}

// NewRealDocument is a factory method that creates a new RealDocument
func NewRealDocument(content string) *RealDocument {
 return &RealDocument{content: content}
}

// ProtectedDocument is the proxy object that will control access to the real object
type ProtectedDocument struct {
 content string
 document *RealDocument
 userRole string
}

// Display is the version that the proxy implements
func (p *ProtectedDocument) Display() string {
 fmt.Printf("Access attempt by %s role\n", p.userRole)
 if p.userRole != "admin" {
 fmt.Println("❌ Access Denied: Insufficient permissions")
 return "Access Denied"
 }
 fmt.Println("✅ Document access granted")
 if p.document == nil {
 fmt.Println("⌛ Loading document content...")
 p.document = NewRealDocument(p.content)
 }
 return p.document.Display()
}

// NewProtectedDocument is a factory method that creates a new ProtectedDocument
func NewProtectedDocument(content string, userRole string) *ProtectedDocument {
 return &ProtectedDocument{content: content, userRole: userRole}
}

func ShowDocument(doc Displayable) {
 fmt.Println(doc.Display())
}

func main() {
 doc := NewRealDocument("Top secret stuff.")

 adminProxy := NewProtectedDocument(doc.content, "admin")
 ShowDocument(adminProxy)

 userProxy := NewProtectedDocument(doc.content, "read-only")
 ShowDocument(userProxy)

 // Whoever has access to the real object can bypass the proxy
 fmt.Println("By-passing the proxy:")
 ShowDocument(doc)

 // Output:
 /*
 Access attempt by admin role
 ✅ Document access granted
 ⌛ Loading document content...
 Displaying Document: Top secret stuff.
 Access attempt by read-only role
 ❌ Access Denied: Insufficient permissions
 Access Denied
 By-passing the proxy:
 Displaying Document: Top secret stuff.
 */
}

Behavioral Design Patterns

These patterns deal with algorithms and the assignment of responsibilities between objects.

Chain of Responsibility (CoR, chain of command)

Analogy
- A call center answers with a machine, then a generalist tries to solve the issue, then the engineer comes to the rescue.
When to use it
- When an action must pass a series of checks, such that a handler may process the request or pass it down the chain of handlers.
Implementation
- A Handler interface defines the interface for handling requests and may also have a method to set the next handler of the chain.
- Concrete handlers implement the Handler interface. These either process the request if they can or pass it down the chain until no more handler is available.
- The Client initiates the request to a chain of handlers. The chain can either be pre-defined or the client might do the chaining.
Pros
- Can control the order of request handling.
- Can decouple unrelated operations (single responsibility principle). It decouples the sender from the receiver.
- New handlers can be added without breaking existing code (open/closed principle).
Cons
- Some requests may not be handled if there are no suitable handlers in the chain.
  - When all requests should be handled, a default “catch-all” handler can be implemented to be used at the end of the chain.
  - Another approach could be to disqualify any request that does not match certain criteria if possible, and stop the chain right at the beginning.
- Performance could be an issue if the chain of handlers is lengthy.
Relation to other patterns
- Chain of Responsibility passes a requests in a sequential manner until a handler deals with it.
- It is often used with the Composite pattern, e.g. in a GUI application where clicking on a button inside a dialog, which itself is inside a panel, may produce a chain reaction. This is suitable when the data structure forms a tree.
- The handlers may be implemented as Commands. Or the request itself may arrive as a Command object, where the same operation could be executed in a series of different contexts.
- Like Decorator, this pattern relies on recursive composition. But a Decorator doesn’t stop at any point in the chain and adds behavior on top of an existing object (adds responsibilities) while Chain of Responsibility can execute any operation in any of its handlers (passes responsibility along).

 go
 
 
 
package main

import "fmt"

// BaseLogger is the base struct for all loggers
type BaseLogger struct {
 next Logger
}

// Logger is the interface that all loggers must implement
type Logger interface {
 Log(message string, severity int)
 SetNext(logger Logger)
}

// Different types of log levels
const (
 INFO = 1
 DEBUG = 2
 ERROR = 3
)

func (l *BaseLogger) SetNext(next Logger) {
 l.next = next
}

func (l *BaseLogger) Next(message string, severity int) {
 if l.next != nil {
 l.next.Log(message, severity)
 }
}

type LoggerImpl struct {
 BaseLogger
 levels map[int]bool // Map to hold allowed log levels
}

func NewLogger(levels ...int) *LoggerImpl {
 levelMap := make(map[int]bool)
 for _, level := range levels {
 levelMap[level] = true
 }
 return &LoggerImpl{levels: levelMap}
}

func (l *LoggerImpl) Log(message string, severity int) {
 if l.levels[severity] {
 fmt.Printf("[%s] %s\n", severityToString(severity), message)
 return
 }
 l.Next(message, severity)
}

func severityToString(severity int) string {
 switch severity {
 case INFO:
 return "INFO"
 case DEBUG:
 return "DEBUG"
 case ERROR:
 return "ERROR"
 default:
 return "UNKNOWN"
 }
}

// DefaultLogger is a logger that logs all unhandled messages
type DefaultLogger struct{}

// Log logs the message and severity level for all unhandled messages
func (l *DefaultLogger) Log(message string, severity int) {
 fmt.Printf("[DEFAULT] %s (severity: %d)\n", message, severity)
}

// SetNext does nothing for the default logger
func (l *DefaultLogger) SetNext(next Logger) {
 // default catch-all handler with no next logger
}

func BuildChain(loggers ...Logger) Logger {
 for i := 0; i < len(loggers)-1; i++ {
 loggers[i].SetNext(loggers[i+1])
 }
 return loggers[0] // Return the head of the chain
}

func main() {
 infoLogger := NewLogger(INFO)
 debugLogger := NewLogger(DEBUG)
 errorLogger := NewLogger(ERROR)
 defaultLogger := &DefaultLogger{}

 // Build the chain
 headLogger := BuildChain(debugLogger, infoLogger, errorLogger, defaultLogger)

 // Test the chain
 headLogger.Log("Info message", INFO)
 headLogger.Log("Debug message", DEBUG)
 headLogger.Log("Error message", ERROR)
 headLogger.Log("Unhandled severity message", 0)

 // Output:
 /*
 [INFO] Info message
 [DEBUG] Debug message
 [ERROR] Error message
 [DEFAULT] Unhandled severity message (severity: 0)
 */
}

Command (action, transaction)

Analogy
- A kitchen chef receives a command.
When to use it
- To encapsulate a request as an object, allowing for the parameterization of clients with queues, requests, and operations.
- Support of undoable operations. It turns a request into a stand-alone object that contains all information about the request. This transformation lets one parameterize methods with different requests, delay or queue a request’s execution, and support undoable operations.
Implementation
- Command Interface: This defines a method for executing a command.
- Concrete Command: Implements the command interface and defines the binding between a Receiver object and an action.
- Receiver: Performs the actual work when the command’s Execute method is called.
- Invoker: Asks the command to carry out the request.
- Client: Creates a ConcreteCommand object and sets its receiver.
Pros
- Decouples the classes that invoke the operation from the object that knows how to execute the operation.
- New commands can be added without changing existing code (Open/Closed Principle).
- Multiple commands can be composed into one by using the Composite pattern.
Cons
- Can introduce additional layers of abstraction, which might complicate simpler operations.
- Each new command might require a new concrete class, increasing the number of classes in the system.
Relation to other patterns
- Memento can be used in conjunction with the Command pattern to implement undo/redo functionalities, where Command stores the state of the Receiver.
- Commands can be composed into a Composite command, a macro command that consists of several simpler commands.
- Command encapsulates a request as an object, whereas Strategy encapsulates an algorithm.

 go
 
 
 
package main

import "fmt"

// Command is the command interface: Redo() is implicit
type Command interface {
 Execute()
 Undo()
}

// BankAccount is the receiver
type BankAccount struct {
 Balance float64
 Difference float64
 Operation string
}

// DepositCommand and WithdrawCommand are the concrete commands
type DepositCommand struct {
 account *BankAccount
 amount float64
}

// Execute adds the amount to the account balance
func (d *DepositCommand) Execute() {
 d.account.Balance += d.amount
 d.account.Difference = d.amount
 d.account.Operation = "deposit"
}

type WithdrawCommand struct {
 account *BankAccount
 amount float64
}

// Execute subtracts the amount from the account balance
func (w *WithdrawCommand) Execute() {
 if w.account.Balance >= w.amount {
 w.account.Balance -= w.amount
 w.account.Difference = w.amount
 } else {
 w.account.Difference = 0
 }
 w.account.Operation = "withdrawal"
}

// Undo reverses the DepositCommand operation
func (d *DepositCommand) Undo() {
 d.account.Balance -= d.amount
 d.account.Difference = d.amount
 d.account.Operation = "undo deposit"
}

// Undo reverses the WithdrawCommand operation
func (w *WithdrawCommand) Undo() {
 w.account.Balance += w.amount
 w.account.Difference = w.amount
 w.account.Operation = "undo withdrawal"
}

// CommandManager is the invoker
type CommandManager struct {
 history []Command
 redoStack []Command
}

// ExecuteCommand adds the command to the history and executes it
func (cm *CommandManager) ExecuteCommand(cmd Command) {
 cmd.Execute()
 cm.history = append(cm.history, cmd)
}

// Undo reverses the last command
func (cm *CommandManager) Undo() {
 if len(cm.history) == 0 {
 return
 }
 cmd := cm.history[len(cm.history)-1]
 cmd.Undo()
 cm.history = cm.history[:len(cm.history)-1] // Remove the last command from history
 cm.redoStack = append(cm.redoStack, cmd) // Add it to the redo stack
}

// Redo re-executes the last undone command
func (cm *CommandManager) Redo() {
 if len(cm.redoStack) == 0 {
 return
 }
 cmd := cm.redoStack[len(cm.redoStack)-1]
 cmd.Execute()
 cm.redoStack = cm.redoStack[:len(cm.redoStack)-1] // Remove the last command from redoStack
 cm.history = append(cm.history, cmd) // Add it back to the history
}

// PrintSummary shows the account balance and the difference after the operation
func (b *BankAccount) PrintSummary() {
 if b.Operation == "" {
 fmt.Printf("Account balance: $%.2f\n", b.Balance)
 return
 }

 if b.Operation == "withdrawal" && b.Difference == 0 {
 fmt.Printf("Withdrawal not successful. Account balance: $%.2f\n", b.Balance)
 return
 }

 if b.Operation == "deposit" && b.Difference == 0 {
 fmt.Printf("Deposit not successful. Account balance: $%.2f\n", b.Balance)
 return
 }

 if b.Operation == "withdrawal" {
 fmt.Printf("Account balance after %s: $%.2f ($%.2f - $%.2f)\n", b.Operation, b.Balance, b.Balance+b.Difference, b.Difference)
 return
 }

 if b.Operation == "deposit" {
 fmt.Printf("Account balance after %s: $%.2f ($%.2f + $%.2f)\n", b.Operation, b.Balance, b.Balance-b.Difference, b.Difference)
 return
 }

 if b.Operation == "undo withdrawal" {
 fmt.Printf("Account balance after %s: $%.2f ($%.2f + $%.2f)\n", b.Operation, b.Balance, b.Balance-b.Difference, b.Difference)
 return
 }

 if b.Operation == "undo deposit" {
 fmt.Printf("Account balance after %s: $%.2f ($%.2f - $%.2f)\n", b.Operation, b.Balance, b.Balance+b.Difference, b.Difference)
 return
 }
}

func main() {
 account := &BankAccount{Balance: 100.0}
 manager := CommandManager{}

 deposit := &DepositCommand{account: account, amount: 50}
 withdraw := &WithdrawCommand{account: account, amount: 30}

 account.PrintSummary()

 manager.ExecuteCommand(deposit)
 account.PrintSummary()

 manager.ExecuteCommand(withdraw)
 account.PrintSummary()

 fmt.Println("Undoing the last operation...")
 manager.Undo()
 account.PrintSummary()

 fmt.Println("Redoing the last operation...")
 manager.Redo()
 account.PrintSummary()

 fmt.Println("Undoing the last operation...")
 manager.Undo()
 account.PrintSummary()

 fmt.Println("Undoing the last operation...")
 manager.Undo()
 account.PrintSummary()

 // Output:
 /*
 Account balance: $100.00
 Account balance after deposit: $150.00 ($100.00 + $50.00)
 Account balance after withdrawal: $120.00 ($150.00 - $30.00)
 Undoing the last operation...
 Account balance after undo withdrawal: $150.00 ($120.00 + $30.00)
 Redoing the last operation...
 Account balance after withdrawal: $120.00 ($150.00 - $30.00)
 Undoing the last operation...
 Account balance after undo withdrawal: $150.00 ($120.00 + $30.00)
 Undoing the last operation...
 Account balance after undo deposit: $100.00 ($150.00 - $50.00)
 */
}

Iterator

Analogy
- All roads lead to Rome. There’s more than one way of iterating one’s path through the city.
When to use it
- When there’s a need to provide a way to access a collection of objects in a sequential manner without the need to understand or expose the underlying structure of the collection.
Implementation
- The Iterator Interface defines the operations required for iterating over a collection, such as getNext(), hasNext(), and sometimes remove().
- The Concrete Iterator implements the Iterator Interface and keeps track of the current position in the collection.
- The Aggregate Interface defines the method to create an iterator.
- The Concrete Aggregate implements the Aggregate Interface and returns an instance of the Concrete Iterator.
Pros
- Allows the traversal of elements without exposing the underlying structure (array, tree, graph, etc.).
- Enables multiple simultaneous traversals of a collection.
- Provides a uniform interface for traversing different types of collections.
Cons
- Can increase the complexity of an application by requiring additional classes and interfaces.
- Introduces some overhead to the traversal process, especially if the iterator is feature-rich (like supporting bidirectional traversal).
Relation to other patterns
- Iterators are often used to traverse Composite trees.
- With Factory Method, it can be used to define a method in the Aggregate Interface for creating an iterator.
- An Iterator can use a Memento to capture the state of an iteration.

 go
 
 
 
package main

import (
 "fmt"
 "slices"
 "strconv"
)

// Book struct represents a book in a collection
type Book struct {
 Title string
 Pages int
 Index string // shows the index of the book in the collection
}

// BookIterator defines the interface for iterating over books
type BookIterator interface {
 HasNext() bool
 Next() *Book
 HasPrevious() bool // not always there: could be added
 Previous() *Book // not always there: could be added
 Count() int // The number of books iterated over
 CurrentIndex() int // The current index
 Remove(direction string) error // implement safe removal
 Reset() // Reset the iterator
}

// BookCollection holds a collection of books
type BookCollection struct {
 Books []*Book
}

// Iterator returns a new BookIterator to iterate over the collection
func (bc *BookCollection) Iterator() BookIterator {
 return &BookCollectionIterator{
 bookCollection: bc,
 currentIndex: -1,
 count: 0,
 }
}

// BookCollectionIterator is the concrete iterator for BookCollection
type BookCollectionIterator struct {
 bookCollection *BookCollection
 currentIndex int
 count int // The number of books iterated over
 iterationStarted bool // Flag to indicate if iteration has started
}

// HasNext checks if there is a next book in the collection
func (it *BookCollectionIterator) HasNext() bool {
 if !it.iterationStarted {
 it.currentIndex = -1
 it.iterationStarted = true
 }
 return it.currentIndex+1 < len(it.bookCollection.Books)
}

// HasPrevious checks if there is a previous book in the collection
func (it *BookCollectionIterator) HasPrevious() bool {
 if !it.iterationStarted {
 it.currentIndex = len(it.bookCollection.Books) // Set to length for backward iteration start
 it.iterationStarted = true
 }
 return it.currentIndex-1 >= 0
}

// Next returns the next book in the collection
func (it *BookCollectionIterator) Next() *Book {
 if it.bookCollection == nil || len(it.bookCollection.Books) == 0 {
 fmt.Println("No books in the collection")
 return nil
 }
 if it.HasNext() {
 it.currentIndex++
 it.count++
 return it.bookCollection.Books[it.currentIndex]
 }
 return nil
}

// Previous returns the previous book in the collection
func (it *BookCollectionIterator) Previous() *Book {
 if it.bookCollection == nil || len(it.bookCollection.Books) == 0 {
 fmt.Println("No books in the collection")
 return nil
 }
 if it.currentIndex < 0 {
 it.currentIndex = len(it.bookCollection.Books)
 }
 if it.HasPrevious() {
 it.currentIndex--
 it.count++
 return it.bookCollection.Books[it.currentIndex]
 }
 return nil
}

// Count returns the number of books iterated over
func (it *BookCollectionIterator) Count() int {
 return it.count
}

// CurrentIndex returns the current index
func (it *BookCollectionIterator) CurrentIndex() int {
 return it.currentIndex
}

// Remove safely removes the current element from the collection
func (it *BookCollectionIterator) Remove(direction string) error {
 if it.currentIndex < 0 || it.currentIndex >= len(it.bookCollection.Books) {
 return fmt.Errorf("remove operation out of bounds")
 }

 // Remove the element at currentIndex
 it.bookCollection.Books = slices.Delete(it.bookCollection.Books, it.currentIndex, it.currentIndex+1)

 if direction == "backward" {
 // Stay at the same index after removal
 } else if direction == "forward" {
 // Decrement currentIndex to adjust for the shift in elements after removal
 it.currentIndex--
 } else {
 panic("Invalid direction")
 }
 return nil
}

func (it *BookCollectionIterator) Reset() {
 fmt.Println("Resetting the iterator")
 it.currentIndex = -1
 it.iterationStarted = false
}

func main() {
 collection := &BookCollection{
 Books: []*Book{
 {Title: "The Go Programming Language", Pages: 380, Index: "0"},
 {Title: "Go in Action", Pages: 325, Index: "1"},
 {Title: "Go Design Patterns", Pages: 246, Index: "2"},
 },
 }

 fmt.Println("Book collection:")
 for i, book := range collection.Books {
 index := strconv.Itoa(i + 1)
 fmt.Println(index+".", book.Title, "-", book.Pages, "pages")
 }

 iterator := collection.Iterator()

 fmt.Println("\nIterating forward, removing big books")
 for iterator.HasNext() {
 book := iterator.Next()
 fmt.Println("➡️", "("+book.Index+")", book.Title, "has", book.Pages, "pages, count:", iterator.Count())

 if book.Pages > 350 {
 fmt.Println("‼️", book.Title, "is a large book")
 if err := iterator.Remove("forward"); err != nil {
 fmt.Println("Remove operation failed:", err)
 } else {
 fmt.Println("❌ ", book.Title, "removed")
 }
 }
 }
 iterator.Reset()

 fmt.Println("\nIterating backward, removing small books")
 for iterator.HasPrevious() {
 book := iterator.Previous()
 fmt.Println("⬅️", "("+book.Index+")", book.Title, "still has", book.Pages, "pages, count:", iterator.Count())

 if book.Pages < 250 {
 fmt.Println("‼️", book.Title, "is a small book")
 if err := iterator.Remove("backward"); err != nil {
 fmt.Println("Remove operation failed:", err)
 } else {
 fmt.Println("❌", book.Title, "removed")
 iterator.Next() // Go forward one step, so we don't skip a book
 }
 }
 }
 iterator.Reset()

 fmt.Println("\nIterating forward again")
 for iterator.HasNext() {
 book := iterator.Next()
 fmt.Println("➡️", "("+book.Index+")", book.Title, "has", book.Pages, "pages, count:", iterator.Count())
 }

 fmt.Println("\nIterating forward once again without having reset")
 for iterator.HasNext() {
 book := iterator.Next()
 fmt.Println("➡️", "("+book.Index+")", book.Title, "has", book.Pages, "pages, count:", iterator.Count())
 }
 fmt.Println("\nTotal iteration count:", iterator.Count())

 // Output:
 /*
 Book collection:
 1. The Go Programming Language - 380 pages
 2. Go in Action - 325 pages
 3. Go Design Patterns - 246 pages

 Iterating forward, removing big books
 ➡️ (0) The Go Programming Language has 380 pages, count: 1
 ‼️ The Go Programming Language is a large book
 ❌ The Go Programming Language removed
 ➡️ (1) Go in Action has 325 pages, count: 2
 ➡️ (2) Go Design Patterns has 246 pages, count: 3
 Resetting the iterator

 Iterating backward, removing small books
 ⬅️ (2) Go Design Patterns still has 246 pages, count: 4
 ‼️ Go Design Patterns is a small book
 ❌ Go Design Patterns removed
 ⬅️ (1) Go in Action still has 325 pages, count: 5
 Resetting the iterator

 Iterating forward again
 ➡️ (1) Go in Action has 325 pages, count: 6

 Iterating forward once again without having reset

 Total iteration count: 6
 */
}

Mediator (Intermediary, Controller)

Analogy
- The control tower acts as a mediator for aircraft pilots.
When to use it
- To provide a centralized communication medium between different objects in a system.
- To promote loose coupling by keeping objects from referring to each other explicitly, allowing their interaction to be managed by the Mediator.
- Reduce complexities and dependencies between tightly coupled objects communicating directly.
Implementation
- The Mediator Interface defines the interface for communication that the concrete mediator will implement.
- The Concrete Mediator implements the Mediator Interface and coordinates communication between colleague objects. It knows and maintains its colleagues.
- The Colleague Classes are objects that communicate with each other through the Mediator. Each colleague class knows its mediator object and communicates with it, instead of communicating directly with other colleagues.
Pros
- Allows reducing the coupling between classes of objects by centralizing communication in the Mediator.
- Simplifies maintenance and interaction between classes by having a central point that controls and manages communication.
- Centralizes control over how objects interact, making it easier to understand and manage complex communication networks.
Cons
- The mediator can become overly complex (a “God” object) if too many responsibilities are added.
- Centralizing too much logic and control in the mediator can introduce a single point of failure in the system.
- Finding the right balance in responsibilities between the mediator and its colleagues can be challenging.
Relation to other patterns
- The Mediator pattern can be implemented using the Observer pattern, where the mediator acts as an observer and the colleagues as subjects.
- The Mediator can be seen as a counterpart of the Facade pattern. Facade defines a simplified interface to a subsystem, Mediator centralizes communication between components of the system.
- Mediator can work closely with the Command pattern to orchestrate complex operations involving multiple objects.

 go
 
 
 
package main

import (
 "fmt"
 "math/rand"
)

// Mediator interface defines the method for sending messages between colleagues
type Mediator interface {
 Send(message string, colleague Colleague)
 Subscribe(senderName string, receiver []Colleague)
 Unsubscribe(senderName string, receiver []Colleague)
}

// Colleague interface represents a colleague in the mediator pattern
type Colleague interface {
 Notify(message string)
 Name() string
}

// ConcreteMediator implements the Mediator interface and coordinates communication between colleagues
type ConcreteMediator struct {
 colleagues map[string]Colleague
 subscriptions map[string][]Colleague
}

// RegisterColleague adds colleagues to the mediator's list
func (m *ConcreteMediator) RegisterColleague(name string, colleague Colleague) {
 if m.colleagues == nil {
 m.colleagues = make(map[string]Colleague)
 }
 m.colleagues[name] = colleague
}

// Subscribe adds receiver(s) to the list of subscribers for a particular sender
func (m *ConcreteMediator) Subscribe(senderName string, receiver []Colleague) {
 if m.subscriptions == nil {
 m.subscriptions = make(map[string][]Colleague)
 }
 for _, r := range receiver {
 m.subscriptions[senderName] = append(m.subscriptions[senderName], r)
 }
}

// Unsubscribe removes receiver(s) from the list of subscribers for a particular sender
func (m *ConcreteMediator) Unsubscribe(senderName string, receiver []Colleague) {
 // print the list of receivers about to be removed, converting the receiver slice to a string
 unsub := ""
 for _, r := range receiver {
 unsub += r.Name() + ", "
 }
 unsub = unsub[:len(unsub)-2] // remove the trailing comma and space
 fmt.Println("❌ Unsubscribing", unsub, "from", senderName)

 if m.subscriptions == nil {
 return
 }
 for _, r := range receiver {
 for i, v := range m.subscriptions[senderName] {
 if v == r {
 m.subscriptions[senderName] = append(m.subscriptions[senderName][:i], m.subscriptions[senderName][i+1:]...)
 }
 }
 }
}

// Send sends messages between colleagues, avoiding notifying the sender
func (m *ConcreteMediator) Send(message string, sender Colleague) {
 senderName := sender.Name()
 for _, receiver := range m.subscriptions[senderName] {
 if receiver != sender { // Avoid sending message to self
 receiver.Notify(message)
 }
 }
}

// ComponentAlice is a colleague that can send and receive messages through the mediator
type ComponentAlice struct {
 mediator Mediator
}

// Notify handles messages sent to ComponentAlice
func (c *ComponentAlice) Notify(message string) {
 fmt.Println("👂 Alice heard:", message)
}

// Send sends a message via the mediator
func (c *ComponentAlice) Send(message string) {
 fmt.Println("🔉 Alice talks:", message)
 c.mediator.Send(message, c)
}

func (c *ComponentAlice) Name() string {
 return "Alice"
}

// ComponentBob is another colleague similar to ComponentAlice
type ComponentBob struct {
 mediator Mediator
}

// Notify handles messages sent to ComponentBob
func (c *ComponentBob) Notify(message string) {
 fmt.Println("🦻 Bob heard:", shuffleString(message))
}

// Send sends a message via the mediator
func (c *ComponentBob) Send(message string) {
 fmt.Println("📣 Bob talks:", shuffleString(message))
 c.mediator.Send(message, c)
}

func (c *ComponentBob) Name() string {
 return "Bob"
}

// ComponentCharlie is another colleague similar to ComponentAlice
type ComponentCharlie struct {
 mediator Mediator
}

// Notify handles messages sent to ComponentCharlie
func (c *ComponentCharlie) Notify(message string) {
 fmt.Println("👂 Charlie heard:", message)
}

// Send sends a message via the mediator
func (c *ComponentCharlie) Send(message string) {
 fmt.Println("🔉 Charlie talks:", message)
 c.mediator.Send(message, c)
}

func (c *ComponentCharlie) Name() string {
 return "Charlie"
}

// ComponentDave is another colleague similar to the others
type ComponentDave struct {
 mediator Mediator
}

// Notify handles messages sent to ComponentDave
func (c *ComponentDave) Notify(message string) {
 fmt.Println("👂 Dave heard:", message)
}

// Send sends a message via the mediator
func (c *ComponentDave) Send(message string) {
 fmt.Println("🔉 Dave talks:", message)
 c.mediator.Send(message, c)
}

func (c *ComponentDave) Name() string {
 return "Dave"
}

func shuffleString(s string) string {
 // Convert string to a slice of runes to handle multi-byte characters
 r := []rune(s)

 rand.Shuffle(len(r), func(i, j int) {
 r[i], r[j] = r[j], r[i]
 })

 return string(r)
}

func main() {
 mediator := &ConcreteMediator{}

 alice := &ComponentAlice{mediator}
 bob := &ComponentBob{mediator}
 charlie := &ComponentCharlie{mediator}
 dave := &ComponentDave{mediator}

 mediator.RegisterColleague("Alice", alice)
 mediator.RegisterColleague("Bob", bob)
 mediator.RegisterColleague("Charlie", charlie)
 mediator.RegisterColleague("Dave", dave)

 // Nobody is subscribed to anyone yet, so no messages will be received
 alice.Send("Anyone here?")

 mediator.Subscribe("Alice", []Colleague{bob, charlie, dave}) // Alice sends to them
 mediator.Subscribe("Bob", []Colleague{alice, charlie}) // Bob sends to them
 mediator.Subscribe("Charlie", []Colleague{alice, bob}) // Charlie sends to them
 mediator.Subscribe("Dave", []Colleague{alice}) // Dave sends to Alice

 alice.Send("Hi, I'm Alice!")
 bob.Send("Hi, I'm Bob!")
 charlie.Send("Hi, I'm Charlie!")
 dave.Send("Hi, I'm Dave!")

 mediator.Unsubscribe("Alice", []Colleague{bob, charlie}) // Alice no longer sends to them
 alice.Send("I'm outta here!")

 // Output:
 /*
 🔉 Alice talks: Anyone here?
 🔉 Alice talks: Hi, I'm Alice!
 🦻 Bob heard: ,ciAleI'H mi !
 👂 Charlie heard: Hi, I'm Alice!
 👂 Dave heard: Hi, I'm Alice!
 📣 Bob talks: i B'!bIHmo,
 👂 Alice heard: Hi, I'm Bob!
 👂 Charlie heard: Hi, I'm Bob!
 🔉 Charlie talks: Hi, I'm Charlie!
 👂 Alice heard: Hi, I'm Charlie!
 🦻 Bob heard: ' aiemH !CIli,hr
 🔉 Dave talks: Hi, I'm Dave!
 👂 Alice heard: Hi, I'm Dave!
 ❌ Unsubscribing Bob, Charlie from Alice
 🔉 Alice talks: I'm outta here!
 👂 Dave heard: I'm outta here!
 */
}

Memento (Snapshot)

Analogy
- The “undo” and “redo” buttons of a text editor (save and restore state) or the “rollback” feature of a database.
When to use it
- When there is a need to restore an object back to its previous state (e.g. “undo” or “rollback” operations), without violating encapsulation.
Implementation
- Originator: The object whose state needs to be saved and restored.
- Memento: An object that encapsulates the state of the originator at a particular time, which is usually immutable once constructed.
- Caretaker: An object that keeps track of the mementos but does not operate on or examine the contents of a memento. It is responsible for knowing when and why to save and restore the originator’s state.
- In languages supporting nested classes, the Memento can be nested inside the Originator.
  - This way, the caretaker can’t access the memento directly, and the originator can access the memento’s fields directly.
- In other languages like Go which don’t even have classes, the implementation can be based on an intermediate interface.
  - This makes the fields of the memento struct public.
  - There is another approach to restrict access:
    - Originator Interface: This defines the high-level interface for creating a memento and restoring the Originator’s state from a memento. However, it does not expose internal state details.
    - ConcreteOriginator: Implements the Originator interface and contains the internal state that needs to be saved and restored. It knows how to save its internal state to a Memento and restore it, but these Mementos are opaque to the outside world. In Go, this would be defined in a separate package to restrict access to the internal state.
    - Memento Interface: Provides a very restricted interface that might only allow the Memento to be passed around but not altered or viewed (no getters for the state, for instance). In Go, this would be an empty struct to prevent any access to the internal state.
    - ConcreteMemento: Implements the Memento interface and stores the internal state of the ConcreteOriginator. The ConcreteMemento is typically a private or inner class of the ConcreteOriginator, ensuring that only the ConcreteOriginator can access the internal state stored within the Memento. In Go, this would be a struct with unexported fields and it would be present in the same package as the ConcreteOriginator.
    - Caretaker: Manages the mementos without knowing their content or structure. It can request a Memento from the Originator and store it, and it can pass a Memento back to the Originator for state restoration, but it cannot access the state stored inside the Memento.
Pros
- The internal state of the originator is saved externally without breaking encapsulation.
- The originator doesn’t need to keep versions of its state, simplifying its code.
- It enables the implementation of undo mechanisms in a relatively straightforward manner.
Cons
- It can be resource-intensive if the originator’s state is large or if there are many mementos stored (think of GIMP or Photoshop when working with large images and a growing undo stack!).
- It adds complexity to the code, particularly in managing the lifecycle and permissions of mementos.
- The caretaker must manage the lifetimes of mementos carefully to avoid excessive memory use (hidden implementation costs).
Relation to other patterns
- Memento can be used in conjunction with the Command pattern to store state for undoable commands.
- It can store the state of an Iterator, allowing it to return to a specific point in the iteration.
- Both Memento and Prototype involve creating a copy of an object, but for different purposes: Prototype for creating a duplicate object; Memento for saving state.

 go
 
 
 
package main

import "fmt"

// Memento is a snapshot of the originator's state
type Memento struct {
 state string
}

// Originator is the object whose state needs to be saved
type originator struct {
 state string
}

// setState updates the state of the originator
func (o *originator) setState(state string) {
 o.state = state
}

// saveStateToMemento saves the current state to a memento
func (o *originator) saveStateToMemento() *Memento {
 return &Memento{state: o.state}
}

// restoreStateFromMemento restores the state from a memento
func (o *originator) restoreStateFromMemento(m *Memento) {
 o.state = m.state
}

// Caretaker could also be a single stack with an index to move back and forth
type Caretaker struct {
 undoStack []*Memento
 redoStack []*Memento
}

// saveState saves the current state of the originator to the undoStack of the caretaker
func (c *Caretaker) saveState(o *originator) {
 m := o.saveStateToMemento()
 c.undoStack = append(c.undoStack, m) // Save the current state before changing it
}

// changeState changes the state of the originator and saves the current state to the undoStack of the caretaker
func (c *Caretaker) changeState(o *originator, newState string) {
 c.saveState(o) // Save the current state before making a change
 o.setState(newState)
}

// undo restores the state of the originator to the previous state, if any
func (c *Caretaker) undo(o *originator) {
 if len(c.undoStack) == 0 {
 return
 }

 lastIndex := len(c.undoStack) - 1
 memento := c.undoStack[lastIndex]
 c.undoStack = c.undoStack[:lastIndex] // Pop from undoStack

 c.redoStack = append(c.redoStack, o.saveStateToMemento()) // Save the current state to redoStack before undoing
 o.restoreStateFromMemento(memento)
}

// redo restores the state of the originator to the next state, if any
func (c *Caretaker) redo(o *originator) {
 if len(c.redoStack) == 0 {
 return
 }

 lastIndex := len(c.redoStack) - 1
 memento := c.redoStack[lastIndex]
 c.redoStack = c.redoStack[:lastIndex] // Pop from redoStack

 c.undoStack = append(c.undoStack, o.saveStateToMemento()) // Save the current state to undoStack before redoing
 o.restoreStateFromMemento(memento)
}

func main() {
 originator := &originator{}
 caretaker := &Caretaker{}

 // Initial state is saved
 caretaker.saveState(originator)

 // Changes to the originator are made through the Caretaker to ensure states are saved
 caretaker.changeState(originator, "State #1")
 caretaker.changeState(originator, "State #2")
 caretaker.changeState(originator, "State #3")

 fmt.Println("Current State:", originator.state)

 caretaker.undo(originator)
 fmt.Println("State after undo:", originator.state)

 caretaker.undo(originator)
 fmt.Println("State after second undo:", originator.state)

 caretaker.undo(originator)
 fmt.Println("State after third undo:", originator.state)

 caretaker.redo(originator)
 fmt.Println("State after redo:", originator.state)

 caretaker.redo(originator)
 fmt.Println("State after second redo:", originator.state)

 caretaker.redo(originator)
 fmt.Println("State after third redo:", originator.state)

 caretaker.redo(originator)
 fmt.Println("State after fourth redo:", originator.state)

 caretaker.undo(originator)
 fmt.Println("State after undo:", originator.state)

 // Output:
 /*
 Current State: State #3
 State after undo: State #2
 State after second undo: State #1
 State after third undo:
 State after redo: State #1
 State after second redo: State #2
 State after third redo: State #3
 State after fourth redo: State #3
 State after undo: State #2
 */
}

Observer (Event-Subscriber, Listener)

Analogy
- A newspaper publisher (subject) has a list of subscribers (observers) who want to be notified when a new issue is published.
When to use it
- To create a subscription mechanism that allows multiple objects to listen and react to events happening in another object, known as the subject.
- For implementing distributed event handling systems, in scenarios where an object needs to notify other objects about its state changes without being dependent on them.
Implementation
- Subject Interface (Publisher): Defines the interface for attaching, detaching, and notifying observers. It contains a list of subscribers with methods like subscribe, unsubscribe, and notify.
  - Concrete Subject: Maintains the state of the object and notifies observers when the state changes.
- Observer Interface (Subscriber): Defines the interface for receiving updates from the subject (usually just an update method).
  - Concrete Observers: Implements the observer interface and keeps a reference to the subject so that they can automatically update themselves in response to state changes.
Pros
- The subject doesn’t need to know anything about the observers, other than they implement the Observer interface (loose coupling).
- Subscribers/observers can be managed at runtime by the client.
- Changes in the subject are effectively broadcasted to all interested observers.
Cons
- If there are many observers or if the updates are frequent, the system can become overwhelmed. Besides, the order of updates could be unpredictable.
- Care must be taken to properly manage observer registration and deregistration to avoid memory leaks.
Relation to other patterns
- Observer resembles closely to the Mediator pattern. While it can be used to implement distributed event handling systems, Mediator is more focused on centralizing communication between objects.
- The Observer pattern can use the Singleton pattern to implement a central registry of observers.

 go
 
 
 
package main

import (
 "fmt"
 "strconv"
 "sync"
)

// Subject is implemented implicitly by WeatherData: it's not needed here
type Subject interface {
 RegisterObserver(o Observer)
 RemoveObserver(o Observer)
 NotifyObservers()
}

// WeatherConditions is the data that will be sent to the observers
type WeatherConditions struct {
 Humidity float64
 Pressure float64
 Temperature float64
}

// WeatherData is the subject/publisher that will notify the observers/subscribers
type WeatherData struct {
 observers []Observer
 conditions WeatherConditions
}

// RegisterObserver adds an observer to the list of observers
func (w *WeatherData) RegisterObserver(o Observer) {
 w.observers = append(w.observers, o)
}

// RemoveObserver removes an observer from the list of observers
func (w *WeatherData) RemoveObserver(o Observer) {
 for i, observer := range w.observers {
 if observer == o {
 w.observers = append(w.observers[:i], w.observers[i+1:]...)
 break
 }
 }
}

// NotifyObservers sends the data to all the observers
// Using goroutines, this will notify in random order
func (w *WeatherData) NotifyObservers() {
 var wg sync.WaitGroup
 for _, observer := range w.observers {
 wg.Add(1)
 go func(o Observer) {
 defer wg.Done() // Ensure the WaitGroup counter decrements
 o.Update(w.conditions)
 }(observer)
 }
 wg.Wait()
}

// SetMeasurementsAndNotify sets the weather conditions and notifies the observers
func (w *WeatherData) SetMeasurementsAndNotify(conditions WeatherConditions) {
 w.conditions = conditions
 w.NotifyObservers()
}

// Observer is the interface that observers must implement
type Observer interface {
 Update(conditions WeatherConditions)
}

// CurrentConditionsDisplay is an observer that displays the current conditions
type CurrentConditionsDisplay struct {
 weatherData *WeatherData
}

// NewCurrentConditionsDisplay creates a new CurrentConditionsDisplay and registers it with the WeatherData
func NewCurrentConditionsDisplay(weatherData *WeatherData) *CurrentConditionsDisplay {
 display := &CurrentConditionsDisplay{weatherData: weatherData}
 weatherData.RegisterObserver(display)
 return display
}

// Update is called by the WeatherData when the conditions change
func (c *CurrentConditionsDisplay) Update(conditions WeatherConditions) {
 fmt.Printf("[fancy display] Current conditions: %.2fF degrees, %.2f%% humidity, %.2f pressure\n",
 conditions.Temperature,
 conditions.Humidity,
 conditions.Pressure,
 )
}

// TemperatureDisplay is an observer that displays only the temperature
type TemperatureDisplay struct {
 weatherData *WeatherData
}

// NewTemperatureDisplay creates a new TemperatureDisplay and registers it with the WeatherData
func NewTemperatureDisplay(weatherData *WeatherData) *TemperatureDisplay {
 display := &TemperatureDisplay{weatherData: weatherData}
 weatherData.RegisterObserver(display)
 return display
}

// Update is called by the WeatherData when the conditions change
func (t *TemperatureDisplay) Update(conditions WeatherConditions) {
 fmt.Printf("[simple display] Temperature: %s degrees\n", strconv.FormatFloat(conditions.Temperature, 'f', 0, 32))
}

func main() {
 weatherData := &WeatherData{}

 NewCurrentConditionsDisplay(weatherData)
 NewTemperatureDisplay(weatherData)

 weatherData.SetMeasurementsAndNotify(WeatherConditions{Temperature: 70, Humidity: 65, Pressure: 30.4})
 weatherData.SetMeasurementsAndNotify(WeatherConditions{Temperature: 68, Humidity: 70, Pressure: 29.2})

 // Output:
 /*
 [fancy display] Current conditions: 70.00F degrees, 65.00% humidity, 30.40 pressure
 [simple display] Temperature: 70 degrees
 [simple display] Temperature: 68 degrees
 [fancy display] Current conditions: 68.00F degrees, 70.00% humidity, 29.20 pressure
 */
}

State

Analogy
- A smartphone can be in different states: locked, unlocked, charging, etc. For example, when it’s locked, pressing the home button doesn’t do anything, but when it’s unlocked, pressing it goes to the home screen.
When to use it
- To allow an object to alter its behavior when its internal state changes. This pattern encapsulates state-specific behavior within state objects, making an object’s behavior dependent on its state.
- For implementing finite state machines in a more organized and maintainable way (i.e., refactor large if-else/switch constructs).
- When the number of states is large, and the state-specific behavior changes frequently.
Implementation
- State Interface: Defines a common interface for all concrete state classes, encapsulating the state-specific behavior.
- Concrete State Classes: Implement the State interface and provide the implementation for the state-specific behavior.
- Context Class: Maintains an instance of a ConcreteState subclass that defines the current state and delegates state-specific behavior to the current state object.
Pros
- Single Responsibility Principle: Each state is encapsulated in its own class.
- Open/Closed Principle: It is easy to add new states without changing the context or other states.
- Reduces complex conditional logic in the context by encapsulating state-specific behavior.
Cons
- Can lead to an increase in the number of classes and boilerplate code.
- The context must carefully manage state objects and transitions, which can get complex.
- Overkill for a simple state machine which doesn’t change much.
Relation to other patterns
- While Strategy usually changes the algorithm used by the context (objects being unaware of each other), State changes the behavior of the context based on its internal state.
- State can be used to change the behavior of Commands based on the state of the system.

 go
 
 
 
package main

import (
 "fmt"
 "sync"
 "time"
)

// State is the interface for the vending machine states
type State interface {
 CancelTransaction()
 DispenseItem()
 InsertMoney(amount int)
}

// ReadyState allows item dispensing if there's sufficient balance
type ReadyState struct {
 machine *VendingMachine
}

// InsertMoney adds money to the vending machine balance
func (rs *ReadyState) InsertMoney(amount int) {
 fmt.Printf("🪙 Inserted $%d\n", amount)
 rs.machine.balance += amount
}

// DispenseItem dispenses an item if there's sufficient balance
// and the machine has items in stock
func (rs *ReadyState) DispenseItem() {
 if rs.machine.balance < rs.machine.itemPrice {
 fmt.Println("🚨 Tried to dispense... Insufficient balance!")
 return
 }

 rs.machine.setState(rs.machine.busyDispensingState)
 fmt.Println("🥫 Dispensing $2 item...")

 time.Sleep(1 * time.Second / 5) // Simulate the dispensing process

 rs.machine.balance -= rs.machine.itemPrice
 rs.machine.itemCount--

 fmt.Println("✅ Item dispensed.")

 if rs.machine.balance > 0 {
 fmt.Println("💵 Returning change of $" + fmt.Sprint(rs.machine.balance))
 rs.machine.balance = 0
 }

 if rs.machine.itemCount == 0 {
 fmt.Println("0️⃣ Out of stock!")
 rs.machine.setState(rs.machine.outOfStockState)
 } else {
 rs.machine.setState(rs.machine.readyState)
 }
}

// CancelTransaction does nothing when the machine is ready
// and there's no transaction to cancel, and it refunds the balance
// when there's a pending transaction
func (rs *ReadyState) CancelTransaction() {
 if rs.machine.balance > 0 {
 fmt.Printf("❌ Cancelling transaction, returning $%d\n", rs.machine.balance)
 rs.machine.balance = 0
 return
 }
 fmt.Println("⚠️ No transaction to cancel.")
}

// OutOfStockState prevents item dispensing
type OutOfStockState struct {
 machine *VendingMachine
}

// InsertMoney does nothing when the machine is out of stock
func (oos *OutOfStockState) InsertMoney(amount int) {
 fmt.Println("🔴 Machine out of stock. Money insertion is futile.")
}

// DispenseItem does nothing when the machine is out of stock
func (oos *OutOfStockState) DispenseItem() {
 fmt.Println("🔴 Machine out of stock. Unable to dispense.")
}

// CancelTransaction does nothing when the machine is out of stock
func (oos *OutOfStockState) CancelTransaction() {
 fmt.Println("🔴 Machine out of stock. No transaction to cancel.")
}

// BusyDispensingState prevents further item dispensing and money insertion
type BusyDispensingState struct {
 machine *VendingMachine
}

func (bd *BusyDispensingState) BusyDispensing() {
 fmt.Println("🛑 Dispensing...")
}

// InsertMoney returns the money when the machine is busy dispensing
func (bd *BusyDispensingState) InsertMoney(amount int) {
 fmt.Println("🛑 Busy dispensing. Money insertion is futile.")
}

// DispenseItem does nothing when the machine is busy dispensing
func (bd *BusyDispensingState) DispenseItem() {
 fmt.Println("🛑 Already dispensing, be patient!")
}

// CancelTransaction does nothing when the machine is busy dispensing
func (bd *BusyDispensingState) CancelTransaction() {
 fmt.Println("🛑 Busy dispensing. No transaction to cancel.")
}

// VendingMachine is the context for the state pattern
// in which all states are defined
type VendingMachine struct {
 busyDispensingState State
 readyState State
 outOfStockState State
 currentState State
 itemCount int
 balance int
 itemPrice int
}

// NewVendingMachine creates a new vending machine
func NewVendingMachine(itemCount, itemPrice int) *VendingMachine {
 vm := &VendingMachine{
 itemCount: itemCount,
 itemPrice: itemPrice,
 }
 readyState := &ReadyState{machine: vm}
 outOfStockState := &OutOfStockState{machine: vm}
 busyDispensingState := &BusyDispensingState{machine: vm}

 vm.busyDispensingState = busyDispensingState
 vm.readyState = readyState
 vm.outOfStockState = outOfStockState
 vm.setState(readyState)

 return vm
}

// InsertMoney adds money to the vending machine balance
func (vm *VendingMachine) InsertMoney(amount int) {
 vm.currentState.InsertMoney(amount)
}

// DispenseItem dispenses an item if there's sufficient balance
func (vm *VendingMachine) DispenseItem() {
 vm.currentState.DispenseItem()
 if vm.itemCount == 0 {
 vm.setState(vm.outOfStockState)
 }
}

// setState changes the current state of the vending machine
func (vm *VendingMachine) setState(state State) {
 vm.currentState = state
}

func (vm *VendingMachine) CancelTransaction() {
 fmt.Println("❌ Cancelling transaction...")
 vm.currentState.CancelTransaction()
}

func main() {
 vendingMachine := NewVendingMachine(10, 2) // 10 items, $2 each

 fmt.Println("Buying 3 items...")
 vendingMachine.InsertMoney(2)
 vendingMachine.DispenseItem()
 vendingMachine.InsertMoney(2)
 vendingMachine.DispenseItem()
 vendingMachine.InsertMoney(5)
 vendingMachine.DispenseItem()

 fmt.Println("\nTrying to buy an item with insufficient balance...")
 vendingMachine.InsertMoney(1)
 vendingMachine.DispenseItem() // Fails
 vendingMachine.CancelTransaction()

 fmt.Println("\nBuying 1 item with extra money...")
 vendingMachine.InsertMoney(5)
 vendingMachine.DispenseItem()

 fmt.Println("\nTrying to dispense multiple times in quick succession won't break the machine...")
 vendingMachine.InsertMoney(5)
 var wg sync.WaitGroup
 for i := 0; i < 3; i++ {
 wg.Add(1)
 go func() {
 defer wg.Done()
 vendingMachine.InsertMoney(5)
 vendingMachine.DispenseItem()
 vendingMachine.CancelTransaction()
 }()
 wg.Wait()
 }

 // Output:
 /*
 Buying 3 items...
 🪙 Inserted $2
 🥫 Dispensing $2 item...
 ✅ Item dispensed.
 🪙 Inserted $2
 🥫 Dispensing $2 item...
 ✅ Item dispensed.
 🪙 Inserted $5
 🥫 Dispensing $2 item...
 ✅ Item dispensed.
 💵 Returning change of $3

 Trying to buy an item with insufficient balance...
 🪙 Inserted $1
 🚨 Tried to dispense... Insufficient balance!
 ❌ Cancelling transaction...
 ❌ Cancelling transaction, returning $1

 Buying 1 item with extra money...
 🪙 Inserted $5
 🥫 Dispensing $2 item...
 ✅ Item dispensed.
 💵 Returning change of $3

 Trying to dispense multiple times in quick succession won't break the machine...
 🪙 Inserted $5
 🪙 Inserted $5
 🥫 Dispensing $2 item...
 ✅ Item dispensed.
 💵 Returning change of $8
 ❌ Cancelling transaction...
 ⚠️ No transaction to cancel.
 🪙 Inserted $5
 🥫 Dispensing $2 item...
 ✅ Item dispensed.
 💵 Returning change of $3
 ❌ Cancelling transaction...
 ⚠️ No transaction to cancel.
 🪙 Inserted $5
 🥫 Dispensing $2 item...
 ✅ Item dispensed.
 💵 Returning change of $3
 ❌ Cancelling transaction...
 ⚠️ No transaction to cancel.
 */
}

Strategy

Analogy
- Choosing a transportation method in Google Maps: walking; driving; cycling or public transport. Each method is a strategy selected via a button in the UI. The goal is to reach a given destination, but the method of transportation can vary, and so will the algorithm calculating the route.
When to use it
- To define a family of algorithms, encapsulate each one, and make them interchangeable (e.g., different text parsers classes implementing the same methods but providing different results).
- Strategy lets the algorithm vary independently from clients that use it.
- When there are multiple ways of doing something and there’s a need to switch between these methods easily at runtime.
Implementation
- Strategy Interface: This defines a common interface for all concrete strategies to implement.
- Concrete Strategies: Implement the Strategy Interface with specific algorithmic implementations.
- Context Class: Maintains a reference to a strategy object and delegates it the responsibility of executing the algorithm. The context is configured with a ConcreteStrategy object and may define an interface that lets the strategy access its data.
Pros
- Allows dynamically changing the behavior of an object.
- Encapsulates algorithm implementations from the core business logic.
- Promotes the reuse of individual strategies across different contexts.
Cons
- Introduces multiple classes and interfaces, which increases complexity.
- Clients must understand how strategies differ (and be aware of their existence) to select the appropriate one.
Relation to other patterns
- Factory Method is often used with Strategy to instantiate the appropriate ConcreteStrategy.
- State has a similar structure to Strategy. While Strategy changes the behavior of a context, State’s behavior change is more intrinsic to the context’s state.
- Command can be seen as a specialization of Strategy, with the strategy being parameterized with actions to execute.

 go
 
 
 
package main

import "fmt"

// Formatter is the strategy interface
type Formatter interface {
 Format(text string) string
}

// MarkdownFormatter formats text with Markdown syntax
type MarkdownFormatter struct{}

// Format formats text with Markdown syntax
func (m *MarkdownFormatter) Format(text string) string {
 return fmt.Sprintf("**%s**", text)
}

// HTMLFormatter formats text with HTML tags
type HTMLFormatter struct{}

// Format formats text with HTML tags
func (h *HTMLFormatter) Format(text string) string {
 return fmt.Sprintf("<strong>%s</strong>", text)
}

// CustomFormatter formats text with custom syntax
type CustomFormatter struct{}

// Format formats text with custom syntax
func (c *CustomFormatter) Format(text string) string {
 return fmt.Sprintf("[[[ %s ]]]", text)
}

// TextEditor is the context inside which the strategy is used
type TextEditor struct {
 formatter Formatter
}

// SetFormatter sets the formatter
func (t *TextEditor) SetFormatter(formatter Formatter) {
 t.formatter = formatter
}

// PublishText publishes the text using the selected formatter
func (t *TextEditor) PublishText(text string) {
 fmt.Println(t.formatter.Format(text))
}

func main() {
 editor := &TextEditor{}

 // Using Markdown formatter
 markdownFormatter := &MarkdownFormatter{}
 editor.SetFormatter(markdownFormatter)
 editor.PublishText("Choosing the Markdown strategy")

 // Switching to HTML formatter
 htmlFormatter := &HTMLFormatter{}
 editor.SetFormatter(htmlFormatter)
 editor.PublishText("Choosing the HTML strategy")

 // Switching to Custom formatter
 customFormatter := &CustomFormatter{}
 editor.SetFormatter(customFormatter)
 editor.PublishText("Choosing the Custom strategy")

 // Output:
 /*
 **Choosing the Markdown strategy**
 <strong>Choosing the HTML strategy</strong>
 [[[ Choosing the Custom strategy ]]]
 */
}

Template method

Analogy
- A recipe for baking a cake: the steps are roughly the same for all cakes, but the ingredients and some steps can vary.
When to use it
- To define the skeleton of an algorithm in the superclass while letting subclasses override specific steps of the algorithm without changing its structure.
- When multiple subclasses should implement a similar algorithm with some slight differences in certain steps. Hooks can be used to provide optional parts of the algorithm that subclasses might or might not override.
Implementation
- Abstract Class / Interface with Template Method: Defines the template method, which is a method that outlines the algorithm’s steps, some of which can be default implementations (concrete methods) or abstract methods (hooks) that subclasses will override.
- Concrete Class: Implements the abstract methods (hooks) defined in the abstract class, thereby providing specific behavior for the steps that vary between different subclasses.
Pros
- Promotes reusing code by extracting common behavior into a superclass.
- Subclasses can override certain steps of the algorithm without changing its structure.
- The superclass can control the algorithm’s structure while allowing subclass-specific behavior.
Cons
- Requires inheritance. This is not a problem in languages like Java, but it can be a limitation in languages that don’t have classes, such as Go.
- Subclasses can only change certain parts of the algorithm, not the overall structure or order of steps.
Relation to other patterns
- Both Template Method and Strategy can encapsulate algorithms. The key difference is that Strategy uses composition to change parts of the algorithm, while Template Method uses inheritance.
- Factory Method is often used as a step in a Template Method, particularly when creating objects that need to vary by subclass.

 go
 
 
 
package main

import "fmt"

type GameAI interface {
 TakeTurn()
 CollectResources()
 BuildStructures()
 BuildUnits()
 Attack()
}

// BaseAI provides default implementations for some methods and leaves others as hooks.
type BaseAI struct {
 GameAI // Embedding the interface for polymorphism
}

func (ai *BaseAI) TakeTurn() {
 ai.CollectResources()
 ai.BuildStructures()
 ai.BuildUnits()
 ai.Attack()
}

func (ai *BaseAI) CollectResources() {
 fmt.Println("Collecting resources...")
}

func (ai *BaseAI) BuildStructures() {
 // Default implementation: Do nothing.
 // This acts as a hook method that can be overridden.
}

func (ai *BaseAI) BuildUnits() {
 // Hook method
}

type WolfAI struct {
 BaseAI
}

func (orc *WolfAI) BuildStructures() {
 fmt.Println("Building Wolf structures...")
}

func (orc *WolfAI) Attack() {
 fmt.Println("Wolves attacking!")
}

func (orc *WolfAI) BuildUnits() {
 fmt.Println("Training Wolf units...")
}

func main() {
 wolfAI := &WolfAI{}
 wolfAI.BaseAI.GameAI = wolfAI // Link for polymorphism

 wolfAI.TakeTurn() // Executes the algorithm, with some steps overridden by WolfAI

 // Output:
 /*
 Collecting resources...
 Wolves attacking!
 */
}

Visitor

Analogy
- A tax auditor visiting different businesses to calculate their taxes.
When to use it
- To separate algorithms from the objects on which they operate, allowing for new operations to be added to these objects without modifying their classes.
- When dealing with a fixed set of object structures but needing to extend their functionality dynamically.
Implementation
- Element Interface: Defines an Accept method that takes a visitor and allows the visitor to perform some operation on the element.
- Concrete Element Classes: Implement the Element Interface and define the Accept method. The Accept method calls the visitor’s method that’s designed to handle the element.
- Visitor Interface: Defines a method for each type of concrete element in the object structure that the visitor can operate on.
- Concrete Visitor Classes: Implement the Visitor interface, providing the implementation for each operation on the concrete elements.
Pros
- Keeps algorithm implementations separate from the classes of the objects they operate on.
- Makes it easy to add new operations without modifying the classes of the elements.
- Each visitor class encapsulates related operations, adhering to the Single Responsibility Principle.
Cons
- Requires changing all visitor classes when a new Element class is added, which can be intrusive.
- Visitors often need access to the private internals of Elements, which can break encapsulation.
Relation to other patterns
- Visitor can be used to apply operations over Composite structures, making it easy to perform operations on complex tree structures.
- Visitor and Iterator can be used together to traverse a complex data structure and perform operations on its elements.
- Visitor is like a powerful version of Command pattern, where the visitor can perform multiple operations on the element.

 go
 
 
 
package main

import (
 "fmt"
 "math"
)

// Shape is the interface that all shapes must implement, which is done
// implicitly here.
type Shape interface {
 Accept(visitor Visitor)
}

// Circle is a shape that has a radius.
type Circle struct {
 Radius float64
}

// Accept allows a visitor to visit the circle.
func (c *Circle) Accept(visitor Visitor) {
 visitor.VisitCircle(c)
}

// Rectangle is a shape that has a width and height.
type Rectangle struct {
 Width, Height float64
}

// Accept allows a visitor to visit the rectangle.
func (r *Rectangle) Accept(visitor Visitor) {
 visitor.VisitRectangle(r)
}

// Visitor is the interface that all visitors must implement
type Visitor interface {
 VisitCircle(*Circle)
 VisitRectangle(*Rectangle)
}

// DrawVisitor is a visitor that can draw shapes.
type DrawVisitor struct{}

// VisitCircle draws a circle.
func (d *DrawVisitor) VisitCircle(c *Circle) {
 fmt.Printf("Drawing a circle with radius: %.2f\n", c.Radius)
}

// VisitRectangle draws a rectangle.
func (d *DrawVisitor) VisitRectangle(r *Rectangle) {
 fmt.Printf("Drawing a rectangle with width: %.2f and height: %.2f\n", r.Width, r.Height)
}

// AreaVisitor is a visitor that can calculate the area of shapes.
type AreaVisitor struct {
 Area float64
}

// VisitCircle calculates the area of a circle.
func (a *AreaVisitor) VisitCircle(c *Circle) {
 a.Area = math.Pi * c.Radius * c.Radius
}

// VisitRectangle calculates the area of a rectangle.
func (a *AreaVisitor) VisitRectangle(r *Rectangle) {
 a.Area = r.Width * r.Height
}

func main() {
 circle := &Circle{Radius: 5}
 rectangle := &Rectangle{Width: 3, Height: 4}

 drawVisitor := &DrawVisitor{}
 circle.Accept(drawVisitor)
 rectangle.Accept(drawVisitor)

 areaVisitor := &AreaVisitor{}
 circle.Accept(areaVisitor)
 fmt.Printf("Circle Area: %.2f\n", areaVisitor.Area)

 rectangle.Accept(areaVisitor)
 fmt.Printf("Rectangle Area: %.2f\n", areaVisitor.Area)

 // Output:
 /*
 Drawing a circle with radius: 5.00
 Drawing a rectangle with width: 3.00 and height: 4.00
 Circle Area: 78.54
 Rectangle Area: 12.00
 */
}

Conclusion

Embarking on the journey through Object-Oriented Programming principles, Software Design Principles, and an array of design patterns, we’ve traversed the most common landscape of structured software design. From the encapsulation and abstraction of OOP to the SOLID foundations of software design, and through the diverse territories of Creational, Structural, and Behavioral patterns, this book summary serves as a compass for developers navigating the complexities of software architecture. Whether you’re refining your craft or laying the cornerstone of your development career, the insights and patterns presented by Alexander offer a blueprint for building software that stands the test of time, adaptable to the ever-evolving demands of technology and user needs. That was a very enjoyable read: recommended!

Resources and references

Book summary: Learning Go – An Idiomatic Approach to Real-World Go Programming

Sébastien Lavoie — Fri, 20 Oct 2023 20:08:00 -0600

Introduction

Go is a modern programming language developed by Google that has gained immense popularity in recent years. It is a statically typed, compiled language that combines aspects of imperative and object-oriented programming. Go aims to provide simplicity, performance, and reliability.

In his book, Jon provides a comprehensive introduction to Go programming. The book focuses on teaching idiomatic Go code by using concrete examples and focusing on how experienced Go developers structure their code. It covers fundamental language features like primitive types, control structures, and composite types. It also dives into more advanced topics like concurrency through goroutines and channels, writing tests, and using reflection.

This summary highlights the key takeaways from each chapter. It aims to provide an overview of the core concepts and best practices for writing clear, readable, robust Go code. It’s written from the perspective of a developer who has experience with other programming languages like Python, Java, and TypeScript but practically none in Go.

Setting Up Your Go Environment

Use golint to enforce the right coding style of a project.
Use go version to check the version of Go installed.
Use go vet to find errors that may not be detected by the compiler, such as having the wrong number of arguments passed to a Printf call.
A common idiom to run multiple commands at once when building a project is to rely on a Makefile like this:

 makefile
 
 
 
.DEFAULT_GOAL := build

fmt:
 go fmt ./...
.PHONY:fmt

lint: fmt
 golint ./...
.PHONY:lint

vet: fmt
 go vet ./...
.PHONY:vet

build: vet
 go build hello.go
.PHONY:build

Typing make will run fmt, then vet, then build since the default task is build, which requires vet to have run first, which in turn requires fmt to have run first, which in turn has no dependency, so fmt runs and the chain continues.

Testing whether a new version of Go works for existing programs compiled on an older version is straightforward:

 bash
 
 
 
go get golang.org/dl/go.1.x.y # replace x.y
go1.x.y download

# try out the changes
go1.x.y build

# If all good, remove this secondary version
go1.x.y env GOROOT
/.../.../go1.x.y
rm -rf $(go1.x.y env GOROOT)
rm $(go env GOPATH)/bin/go1.x.y

# Install new version as wanted...

Predeclared Types and Declarations

Predeclared types

Zero value
- Assigned to a variable that is declared with no initial value (doesn’t lead to bugs like in C or C++).
Literals
- These express different bases, such as 0b (binary), 0o (octal) or 0x (hexadecimal). As in other languages like Python or Java, underscores can be used to express large numbers by separating digits (e.g., 1_000_000).
- Floating point literals look like 6.03e23.
- Rune literals are represented with single quotes (no double quotes accepted). The most common ones are ('\n'), tab ('\t'), single quote ('\''), double quote ('\"') and backslash ('\\'). Other bases are supported but should be limited to specific contexts (e.g., bit filters for base two).
- String literals can be written with double quotes, where everything must be escaped.
- Raw string literals use backticks instead of double quotes and can be used to insert any character except a backtick. They support multiline expressions.
Boolean
- The type is bool and the default zero value is false.
Numeric types
- Integers
  - int8 (aliased byte, which is much more common), int16, int32, int64, uint8, uint16, uint32, uint64.
  - The zero value is 0.
  - Use the minimum size when needed for specific applications.
  - Use int64 and uint64 for library functions (suggested back when generics weren’t available).
  - Otherwise, just use int. Other types should be considered a premature optimization until proven otherwise.
  - Variables can be modified like so: +=, -=, *=, /= and %=.
  - Available comparisons are: ==, !=, >, >=, <, and <=.
  - Bit manipulations
    - Shifts: << (left), >> (right)
    - Logical bit masks: & (AND), | (OR), ^ (XOR), &^ (AND NOT)
    - These operators can be used to modify a variable as well: &=, |=, ^=, &^=, <<=, >>=.
Floating point types
- float32, float64.
- The zero value is 0.
- If using a floating point number, opt for float64 unless a profiler shows significant improvement with float32 and the precision is good enough (6-7 decimal places).
- Strict equality (or inequality) should not be done on floating point numbers: check the variance instead (less than epsilon).
Complex types
- complex64 uses float32 to represent real and imaginary parts, while complex128 uses float64, using the complex built-in function and real and image functions to extract the relevant parts.
- As with floating point numbers, use the epsilon technique to check for equality.
Strings and runes
- The zero value is an empty string.
- Strings are immutable.
- Strings can be checked for equality or compared for ordering (>, >=, <, or <=) and can be concatenated with the + operator.
- The rune type represents a single code point, equivalent to int32.
Explicit type conversion
- All type conversions are explicit.
- There is no concept of “truthiness” (e.g., if 2: print("ok") is valid in Python).

`var` vs. `:=`

var is more verbose but flexible:

 go
 
 
 
var x int = 1
var x = 1 // because the default type is `int`
var x int // no value => it will be the zero value
var x, y int = 1, 2 // multiple assignments
var x, y int // multiple assignments, zero values
var x, y = 1, "hi" // different default types

// Declaration list
var (
 x int
 y = 2
 z string
)

Type inference can be performed within a function:

 go
 
// These statements are equivalent
var x = 1
x := 1 // invalid syntax outside a function

Avoid := in the following situations:

When explicitly initializing a zero value, like var x int.
To avoid a type conversion, by writing var x byte = 8 instead of x := byte(8).
To avoid “shadowing” a variable, as := can be used to assign to existing variables. Create new variables with var.
Non-constant package-level variables are a bad idea. If they’re unused, they go unnoticed without raising compile-time errors.

`const`

Variables cannot be declared as immutable.
Constants are a way of giving names to literals.
Inside a function, it is clear when a variable is being modified.
If a constant is typed (e.g., const typedVar int = 1), then it can only be assigned to that type, int in this case.
If a constant is untyped (e.g., const untypedVar = 2), then it can be assigned to suitable numerical types.

Unused variables

Unused declared local variables result in a compile-time error.

Naming Variables and Constants

Even though many Unicode characters can be used, they should be avoided to maintain clarity.
Go uses camelCase.
The less scope a variable has, the shorter its name should be (k and v are accepted for key/value, just like i and j to use indices when iterating in loops).
It is common to use the first letter of a type as the variable name (e.g., i for integers, f for floats, b for boolean). If the code is hard to understand, it’s a sign the function is trying to do too much.

Composite Types

Arrays

They are rarely used directly.
They can be compared (== and !=).
Their length is retrieved with the built-in len function.
Negative indexing is a compile-time error.
Out-of-bounds indexing results in a panic at runtime.
Unless there’s a very specific need to use a given size of array (e.g., for a cryptographic library), avoid them.
They exist basically to provide slices.
- There are a few ways of declaring arrays:

 go
 
 
 
// indicate the size and type
var x [3]int // 3 integers assigned to the zero value

// array literal
var x = [3]int{10, 20, 30} // values specified
var x = [...]int{10, 20, 30} // equivalent

// sparse array:
// indicate few values at specific locations
var x = [12]int{1, 5: 4, 6, 10: 100, 15}

// Get and set values
x[0] = 10
fmt.Println(x[2])

Slices

The zero value for a slice is nil, which represents the lack of a value for some type. nil itself has no type.
The size of the array is not specified, making it a slice: var x = []int{10, 20, 30}. This is a slice literal.
Can be used like a sparse array: var x = []int{1, 5: 4, 6, 10: 100, 15}.
Multidimensional arrays can be simulated: var x [][]int.
Reads and assignments are the same as with arrays, using square brackets.
Slices can be created without assigning initial values: var x []int.
Slices aren’t comparable, except to check if it is nil (x == nil).
They’re useful for sequential data.

`len`

A nil slice returns 0 (len(x)).

`append`

It it used to grow slices:

 go
 
 
 
var x []int
x = append(x, 10) // returns a slice
x = append(x, 5, 6, 7) // more than one value

// append to another slice with `...`
// Similar to the spread operator in JavaScript,
// but it goes after the value to spread
y := []int{20, 30, 40}
x = append(x, y...)

Capacity

It increases automatically as needed. It doubles under 1,024 items, then it increases by at least 25%.
cap returns the current capacity of the slice.
It is better when possible to allocate the needed size upfront to avoid resizing the arrays.

`make`

It can be used to create a slice that already has a capacity specified.
- x := make([]int, 5): length and capacity of 5 (all zero values). Using append here would add new values to the end of the slice, after the zero values!
- x := make([]int, 0, 10) creates an empty slice with a capacity of 10 and after x = append(x, 5,6,7,8), it contains [5 6 7 8].

Emptying a slice

Available since Go 1.21 with the built-in clear function.
This sets all elements of the slice to their zero value.

 go
 
 
 
x := []int{1, 2, 3, 4}
fmt.Println(x) // [1 2 3 4]
clear(x)
fmt.Println(x) // [0 0 0 0]

y := []string{"a", "b", "c"}
fmt.Println(y) // [a b c]
clear(y)
fmt.Println(y) // [ ] (empty strings)

Declaring a slice

Slice literals
- An empty slice literal declares a slice that is non-nil: var x = []int{}. This is useful to convert to JSON.
- Useful with some initial values or when the values don’t change.

Slicing slices

These work with square brackets:

 go
 
 
 
x := []int{1, 2, 3, 4} // [1 2 3 4]
y := x[:2] // [1 2]
z := x[1:] // [2 3 4]
d := x[1:3] // [2 3]
e := x[:] // [1 2 3 4]

 go
 
 
 
x := []int{1, 2, 3, 4}
y := x[:2]
z := x[1:]

// These are bidirectional changes!
x[1] = 20 // affects `x`, `y` and `z`
y[0] = 10 // affects `x` and `y`
z[1] = 30 // affects `x` and `z`

// Result:
// x: [10 20 30 4]
// y: [10 20]
// z: [20 30 4]

append can lead to unintuitive results, overwriting existing values:

 go
 
 
 
x := make([]int, 0, 5) // length 0, capacity 5
x = append(x, 1, 2, 3, 4) // x is now [1 2 3 4]
y := x[:2] // [1 2], length 2, capacity 5
z := x[2:] // [3 4], length 2, capacity 3

y = append(y, 30, 40, 50)
// x is now [1 2 30 40], length 4, capacity 5!
// y is now [1 2 30 40 50], length 5, capacity 5
// z is now [30 40], length 2, capacity 3

x = append(x, 60)
// x is now [1 2 30 40 60], length 5, capacity 5
// y is now [1 2 30 40 60], length 5, capacity 5!
// z is still [30 40], length 2, capacity 3

z = append(z, 70)
// x is now [1 2 30 40 70], length 5, capacity 5!
// y is now [1 2 30 40 70], length 5, capacity 5!
// z is now [30 40 70], length 3, capacity 3

One way to avoid this issue is to use full slice expressions to indicate the capacity of the sub-slices:

 go
 
 
 
x := make([]int, 0, 5)
x = append(x, 1, 2, 3, 4)
y := x[:2:2] // take a slice of x, up to index 2, with a capacity of 2
z := x[2:4:4] // take a slice of x, from index 2 to 4, with a capacity of 2

y = append(y, 30, 40, 50)
// x is still [1 2 3 4], length 4, capacity 5
// y is now [1 2 30 40 50], length 5, capacity 6
// z is still [3 4], length 2, capacity 2

x = append(x, 60)
// x is now [1 2 3 4 60], length 5, capacity 5
// y is still [1 2 30 40 50], length 5, capacity 6
// z is still [3 4], length 2, capacity 2

z = append(z, 70)
// x is still [1 2 3 4 60], length 5, capacity 5
// y is still [1 2 30 40 50], length 5, capacity 6
// z is now [3 4 70], length 3, capacity 4

Converting Arrays to Slices

Arrays can be sliced, though memory will be shared as when slicing a slice.

 go
 
 
 
x := [4]int{5, 6, 7, 8} // [5 6 7 8]
y := x[:2] // [5 6]
z := x[2:] // [7 8]
x[0] = 10 // [10 6 7 8]
// y and z are now [10 6] and [7 8]

Converting Slices to Arrays

Data in the slice is copied to new memory (slices and arrays remain independent).
A slice converted into a pointer to an array will share the same underlying data and memory address.

 go
 
 
 
x := []int{1, 2, 3, 4} // [1 2 3 4]
y := x // y is a slice, not a copy
z := make([]int, 4) // [0 0 0 0]
copy(z, x) // copy(dst, src)
x[0] = 10 // [10 2 3 4]
// y is now [10 2 3 4]
// z is now [1 2 3 4]

`copy`

It creates a slice that is independent from the original slice.

 go
 
 
 
// It can copy the whole slice if the lengths are the same
x := []int{1, 2, 3, 4} // [1 2 3 4]
y := make([]int, 4) // [0 0 0 0]
num := copy(y, x) // num=4, copy(dst, src)
fmt.Println(y, num) // [1 2 3 4] 4
fmt.Println(x) // [1 2 3 4]

// It can copy a subset of the slice
y := make([]int, 2) // [0 0]
num = copy(y, x) // num=2, copy(dst, src)
fmt.Println(y) // [1 2]

// It can copy from a subset from any position
x := []int{1, 2, 3, 4} // [1 2 3 4]
y := make([]int, 2) // [0 0]
copy(y, x[2:]) // copy(dst, src)
fmt.Println(y) // [3 4]

num = copy(x[:3], x[1:]) // put the last 3 values at the beginning
fmt.Println(x) // [2 3 4 4], overwriting

// It also works with arrays
x := []int{1, 2, 3, 4} // slice, [1 2 3 4]
d := [4]int{5, 6, 7, 8} // array, [5 6 7 8]
y := make([]int, 2) // [0 0]
copy(y, d[:]) // first 2 values of d into y
fmt.Println(y) // [5 6]
copy(d[:], x) // copy x into d
fmt.Println(d) // [1 2 3 4]

Strings and Runes and Bytes

Strings are arrays of bytes.
Single characters can be extracted from a string with an index expression:

 go
 
 
 
s := "hello"
c := s[0] // c is a byte, not a rune
fmt.Println(c) // 104
fmt.Printf("%T\n", c) // uint8 (i.e., byte)

// Need to be careful with indexing
var s string = "Hello 😄"
fmt.Println(len(s)) // could have expected 7, but it's 10
fmt.Println(s[:2], s[7:]) // He ���: the emoji is 4 bytes long

Maps

The built-in map type is a hash map (implemented as an array).
The zero value for a map is nil.
Writing to a nil map results in a runtime panic.
len on a map returns the number of key/value pairs.
Maps are not comparable (but they can check against nil).
The key must be comparable: it cannot be a slice, map or function.
Maps are good when the order of the keys doesn’t matter: use a slice when it does.
All the values must be of the same type, but the keys can be of different types.
Avoid using them as input parameters to functions (use a struct instead to be self-documenting).

 go
 
 
 
var nilMap map[string]int
// ^ ^
// key type value type

// map literal: length of 0
myMap := map[string]int{} // allows reads and writes

// Non-empty map
reposByOrg := map[string][]string{
 "dbeaver": []string{"dbeaver", "cloudbeaver", "team-edition-deploy"},
 "slidevjs": []string{"slidev", "slidev-vscode", "themes"},
 "ReactiveX": []string{"RxJava", "rxjs", "RxGo"}, // comma at the end here too
}

// With a default size
myValues := make(map[int][]string, 10) // length 0, capacity 10, can grow beyond 10

Reading and writing maps

 go
 
 
 
reposByOrgStars := map[string]int{} // length 0, can grow, string to integer
reposByOrgStars["dbeaver"] = 10 // write
reposByOrgStars["slidevjs"] = 100 // write
reposByOrgStars["ReactiveX"]++ // read, increment, write (0 -> 1)
// reposByOrgStars["slidevjs"] := 100 // invalid syntax

Comma Ok idiom

One can get the value of a key and a boolean indicating whether the key exists or not:

 go
 
value, ok := reposByOrgStars["dbeaver"] // 10 true
value, ok = reposByOrgStars["notfound"] // 0 false

Deleting from a map

delete removes a key/value pair from a map:

 go
 
 
 
delete(reposByOrgStars, "ReactiveX")

// It is safe to delete a key that doesn't exist
delete(reposByOrgStars, "notfound") // it returns nothing

// It is safe to delete a key from a nil map
var nilMap map[string]int
delete(nilMap, "notfound")

// It is safe to delete a key from an empty map
emptyMap := map[string]int{}
delete(emptyMap, "notfound")

Using maps as sets

Go doesn’t have a built-in set type.
A map can be used as a set by using the key as the value and the value as a boolean:

 go
 
 
 
mySet := map[string]bool{}
mySet["hello"] = true
mySet["world"] = true
mySet["hello"] = true // no error, but it's still a set

This works because if the value isn’t found, the zero value is returned, which is false for booleans.
To use operations like union, intersection and difference, the most convenient solution is to use a third-party library.
Structs can also be used as sets as they’re more memory efficient, but more clumsy to use as they make use of the comma ok idiom.

Structs

Good when there is related data that needs to be grouped together.
They are defined with the type keyword.
No commas are needed between fields.
They can be defined inside or outside of a function. If inside a function, they can only be used inside that function.

 go
 
 
 
type user struct {
 age int
 firstName string
 lastName string
}

var u user // zero value for a struct is all zero values for its fields
u.age = 18
u.firstName = "Bob"
u.lastName = "Michigan"
fmt.Println(u) // {18 Bob Michigan}

// Assignments can also be done with a struct literal
bob := user{} // also initializes all fields to zero values

// With initial values
jeremy := user{
 49, // must match the order of the fields
 "Jeremy", // all fields must be specified
 "Stretchy",
} // {49 Jeremy Stretchy}

// With this style, fields can be left out:
otherUser := user{
 age: 32, // using the field name...
 firstName: "Sweaty", // cannot indicate other fields without the field name
 // lastName is zero value, i.e., empty string
}

// Accessing a field:
jeremy.firstName = "Jer"
fmt.Println(jeremy.firstName) // Jer

Anonymous structs

They are useful when a struct is only used in one place.
Useful when marshalling and unmarshalling data.

 go
 
 
 
computer := struct {
 operatingSystem string
 chip string
}{
 operatingSystem: "macOS",
 chip: "Apple M2 Ultra",
}

Comparing and converting structs

Contrary to regular structs, they can be compared.

Blocks, Shadows, and Control Structures

Blocks

A block is a place where declarations are made.
The top-level block is the package block.
The import statements are in the file block.

Shadowing Variables

There is a “global” block, the universe block, which contains the built-in functions and types. Careful: those keywords can be shadowed!

 go
 
 
 
func main() {
 x := 10
 if x > 5 {
 fmt.Println(x)
 x := 5 // shadowing
 fmt.Println(x)
 }
 // x is still 10 here (outer block)
 fmt.Println(x)
}

func main() {
 x := 10
 if x > 5 {
 // x is shadowed while y is declared!
 x, y := 5, 20
 fmt.Println(x, y) // 5 20
 }
 fmt.Println(x) // 10
}

func main() {
 x := 10
 fmt.Println(x)
 fmt := "shadowing fmt package"
 fmt.Println(fmt) // undefined!
}

Linters typically won’t catch shadowing, but we can install shadow:

bash

go install golang.org/x/tools/go/analysis/passes/shadow/cmd/shadow@latest

And we can make it part of the Makefile:

 makefile
 
 
 
lint: fmt
 golint ./...
 shadow ./...
.PHONY: lint

`if` statements

The condition must be a boolean expression.
The usual flow is if ... {...} else if ... { ... } else { ... }.
There are no parentheses around the condition.
Variables can be scoped to the if statement (they’ll be available in else if and else blocks as well). Only use that feature to define new variables!

 go
 
 
 
if n := rand.Intn(10); n == 0 {
 fmt.Println("That's too low")
} else if n > 5 {
 fmt.Println("That's too big:", n)
} else {
 fmt.Println("That's a good number:", n)
}
// n is not available here!

`for` loops

It is the only looping construct in the language.
It can be used in four different ways to accomplish all looping needs.
It uses no parentheses.

Complete `for` statement

 go
 
for i := 0; i < 10; i++ {
 fmt.Println(i)
}

The initialization statement is executed once before the loop starts. It must use the := operator. It can shadow variables.
The second statement is the condition. It must be a boolean expression. It is checked before each iteration as well as before the loops starts.
The third statement is the post statement. It is executed after each iteration. It would usually be used to increment a counter.

Condition-only `for` Statement

 go
 
 
 
i := 0
for i < 10 {
 fmt.Println(i)
 i++
}

This is a statement that only has a condition. It is equivalent to a while loop in other languages.

Infinite `for` Statement

 go
 
for {
 fmt.Println("Hello")
}

This is a statement that has no condition. It is equivalent to a while true loop in other languages.

`break` and `continue`

There is no do... while construct as in other languages like C, Java or JavaScript.
do... while indicates how to stay in the loop while for indicates how to leave the loop.
break exits the loop.
continue skips the rest of the loop and goes to the next iteration. Use it to avoid nesting loops.
break and continue can be used with labels to break out of nested loops.

`for-range` loop

 go
 
 
 
odds := []int{1, 3, 5, 7, 9}
for i, v := range odds {
 fmt.Println(i, v)
}

Here, i (0 to 4) is the index of the iterator construct and v (1, 3 ... 9) is the value.
i is usually used for arrays, slices and strings, while k is used for maps.
As in other languages like Python, use an underscore (_) to ignore a value. The index or key can be ignored this way.
If the key is needed but not the value, leave the value out: for k := range myMap { ... }. This is useful when using a map as a set.

Iterating over maps

 go
 
 
 
m := map[string]int{"a": 1, "c": 3, "b": 2}
for i := 0; i < 3; i++ {
 fmt.Println("Loop", i)
 for k, v := range m {
 fmt.Println(k, v) // order will differ
 }
}
fmt.Println(m) // will be in ascending order to help with debugging

For security reasons (hash DoS), the order of the keys is randomized. If the order is important, use a slice of keys instead.

Iterating over strings

Go iterates over the Unicode code points (runes), not the bytes.

 go
 
 
 
samples := []string{"hello", "worl∂!"}
for _, sample := range samples {
 for i, r := range sample {
 fmt.Println(i, r, string(r))
 }
 fmt.Println()
}

/*
Output:
0 104 h // index, rune, string(rune)
1 101 e
2 108 l
3 108 l
4 111 o
 // first string is 5 runes long
0 119 w
1 111 o
2 114 r
3 108 l
4 8706 ∂ // this is a single rune: it occupies 3 bytes
7 33 ! // index 7! The second string is 8 runes long
*/

`for-range` value is a copy

Modifying the value variable doesn’t change the original value.
break and continue are also available in this form.

 go
 
 
 
odds := []int{1, 3, 5, 7, 9}
for _, v := range odds {
 v += 2
}
fmt.Println(odds) // [1 3 5 7 9]

`for` loop labels

While their use is rare, they can be used to break out of nested loops.
The label is indented to the same level as the containing block.

 go
 
 
 
 samples := []string{"hello", "worl∂!"}
outer:
 for _, sample := range samples {
 for i, r := range sample {
 fmt.Println(i, r, string(r))
 if r == 'l' {
 continue outer
 }
 }
 fmt.Println()
 }

/* Output:
0 104 h
1 101 e
2 108 l
0 119 w
1 111 o
2 114 r
3 108 l
*/

This code will skip the remaining letters of both words once the first l is printed out for each one.

The right `for` statement

for-range is the proper way to iterate over strings to get runes.
A complete for statement is good when not iterating through all the items in a collection (except for strings since runes aren’t necessarily one byte long).
The condition-only for statement is used to replace while loops.
Infinite loops can simulate a do... while construct and can be used to create the iterator pattern.

Expression `switch` statements

They don’t use parentheses.
Used to check for equality.
Scoped variables can be declared (e.g., word is scoped to all cases).
All cases are part of the same block (only the switch statement itself is a block surrounded by braces).
No need for break statements. It can be used, but may indicate a code smell.
break can be useful if the switch statement is inside a loop.
There is a fallthrough keyword to go to the next case (not recommended).

 go
 
 
 
words := []string{"Go", "Ada", "COBOL", "C++", "Python", "Clojure", "WebAssembly"}
for _, word := range words {
 switch size := len(word); size {
 case 1, 2, 3, 4: // catches multiple matches
 fmt.Println(word, "is a short name!") // Go, Ada, C++
 case 5:
 wordLen := len(word)
 fmt.Println(word, "is the right length:", wordLen) // COBOL
 case 6, 7, 8, 9: // empty case, nothing happens: Python, Clojure
 default:
 fmt.Println(word, "is a long name!") // WebAssembly
 }
}

// Inside a loop
loop: // label
for i := 0; i < 10; i++ {
 switch {
 case i%2 == 0:
 fmt.Println(i, "is even")
 case i%3 == 0:
 fmt.Println(i, "is divisible by 3 but not 2")
 case i%7 == 0:
 fmt.Println("exit the loop!")
 break loop // break out of the loop, not just the switch statement
 default:
 fmt.Println(i, "is boring")
 }
}

Blank `switch` statements

Instead of checking for equality, they check for any boolean comparison.

 go
 
 
 
func main() {
 var x int = 5
 switch {
 case x%2 == 0:
 fmt.Println(x, "is even")
 case x > 6:
 fmt.Println(x, "is large")
 case x <= 6:
 fmt.Println("Got it!") // this is the one that will be executed
 default:
 fmt.Println(x, "Not it...")
 }
}

`if` or `switch`?

switch should be used when there is some relationship between comparable elements. It is more concise and makes the comparisons more obvious.

`goto`

It should generally be avoided.
It can be used to replace boolean flags, such as in this non-trivial example from the standard library.

Functions

Declaring and calling them

Go has no classes, but it has methods (see chapter 7).
Types are mandatory.
The return keyword is mandatory (except for main) if the function has a return type.
Nothing goes between the input parameters and the start of the block if there’s no return type.
Go has no named or optional input parameters: you can pass structs instead. In practice, that probably means the function is trying to do too much if you need this feature.
Go supports variadic parameters (e.g., the fmt.Println function) with ... right before the type: they are used as a slice inside the function.
Functions can return multiple values. They must all be returned, comma-separated. Unlike Python which uses tuples, Go uses the comma to separate the individual values.
The error is always the last parameter a function will return. If no error occurred, it will be nil.
Indicate ignored values (possibly all) with an underscore. A notable exception is fmt.Println which returns two values that aren’t usually used.
Named values can be returned. They make shadowing possible and should be used sparingly.
Blank returns can be returned with named values, but they make it harder to understand how data flows.

 go
 
 
 

func div(numerator int, denominator int) int {
// ^ ^ ^ ^
// | input parameter | return type
// Function name parameter type
 if denominator == 0 {
 return 0
 }
 return numerator / denominator
}

// The following is equivalent when the types are the same
// func div(numerator, denominator int) int ...

// variadic parameter
func addTo(base int, vals ...int) []int {
 out := make([]int, 0, len(vals))
 for _, v := range vals {
 out = append(out, base+v)
 }
 return out
}
// fmt.Println(addTo(1, 2, 3, 4))
// a := []int{4, 3}
// fmt.Println(addTo(3, a...))
// fmt.Println(addTo(3, []int{4, 5}...))

// named return values
func divAndRemainder(numerator int, denominator int) (result int, remainder int, err error) { ... }

Functions are values

Functions can be defined as types, e.g., type aFuncType func(int, int) int.
Anonymous functions can be defined inside other functions and called immediately (IIFE). This comes in handy when using defer and Goroutines.

Closures

Closures are used to create functions that have access to variables that are outside of their scope.
Functions can be passed as parameters to other functions.
Functions can return functions.
They are useful with sort.Search and sort.Slice.

An example of closure is with the sort.Slice function:

 go
 
 
 
import (
 "fmt"
 "sort"
)

func main() {
 type Person struct {
 FirstName string
 LastName string
 Age int
 HasPet bool
 }
 people := []Person{
 {"Pat", "Patterson", 37, true},
 {"Tracy", "Bobbert", 23, false},
 {"Fred", "Fredson", 18, true},
 {"Bob", "Tracier", 18, false},
 {"Alice", "Anderson", 30, true},
 }
 fmt.Println(people)

 // `people` is captured by the closure
 sort.Slice(people, func(i int, j int) bool {
 // If one person has a pet and the other doesn't,
 // prioritize the one with the pet.
 if people[i].HasPet != people[j].HasPet {
 return people[i].HasPet
 }
 // If both have pets, sort by LastName.
 if people[i].HasPet && people[j].HasPet {
 return people[i].LastName < people[j].LastName
 }
 // Otherwise, sort by age in ascending order.
 return people[i].Age < people[j].Age
 })

 fmt.Println(people)
}

// This returns:
// - People that have pets first
// - If both have pets, sort by last name in ascending order
// - If neither have pets, sort by age in ascending order

// Unsorted:
[
 {Pat Patterson 37 true}
 {Tracy Bobbert 23 false}
 {Fred Fredson 18 true}
 {Bob Tracier 18 false}
 {Alice Anderson 30 true}
]

// Sorted:
[
 {Alice Anderson 30 true}
 {Fred Fredson 18 true}
 {Pat Patterson 37 true}
 {Bob Tracier 18 false}
 {Tracy Bobbert 23 false}
]

A function that returns a function:

 go
 
 
 
func main() {
 withTwo := getResult(2)
 withThree := getResult(3)
 for i := 0; i < 3; i++ {
 fmt.Println("i=", i, "withTwo (2 + i):", withTwo(i), "withThree (3 + i):", withThree(i))
 }
}

func getResult(initialValue int) func(int) int {
 return func(subsequent int) int {
 return initialValue + subsequent
 }
}

// Output:
// i= 0 withTwo (2 + i): 2 withThree (3 + i): 3
// i= 1 withTwo (2 + i): 3 withThree (3 + i): 4
// i= 2 withTwo (2 + i): 4 withThree (3 + i): 5

`defer`

This is used to perform the cleanup code, such as closing a file, after a function has returned. This is similar to the finally block in Java or Python but it executes at the very end of the function, not as part of a try...except block.
It delays the execution of a function until the surrounding function returns.
They run in LIFO (last in, first out) order.
The code that runs after defer is literally the last thing that runs before the function returns, so what is put there is immediately “called” (e.g., defer close()) but will run until later.
It helps reduce depth of nesting, which, along with “lack of structure”, are two of the most important factors that contribute to code complexity (see this paper).

Go is “call by value”

Go always makes a copy of the value before passing it to a function.
Maps and slices behave differently because they are implemented with pointers.

Pointers

A pointer is a variable that holds the address of a value in memory.
The zero value of a pointer is nil.
& is the address-of operator. It goes before the variable name to get its address.
* is the indirection operator (dereference). It goes before a pointer to get the value it points to.
Dereferencing a nil pointer will result in a runtime panic.
Types with an * are pointers to that type (“pointer type”).
new is a built-in function that allocates memory for a type and returns a pointer to it.
To turn a constant into a pointer, use a helper function that takes a value and returns a pointer to it.
A pointer is used to indicate that a parameter is mutable, i.e., that the function can modify the original value.
To update the value of a pointer inside a function, dereference it and assign it a new value.
Value types should be preferred when returning from functions.
Use a pointer as a return type when there is a need to return a modified data structure or when the data being passed around is very large (at least 1 MB).
Slices can be used as buffers when iterating over files to avoid allocating memory in each iteration of the loop.
The garbage collector will free memory that is no longer used. Go favors low latency over high throughput.

 go
 
 
 
var x int = 10
var y *int = &x
fmt.Println(x, y) // 10 0xc0000b4008
fmt.Println(&x, *y) // 0xc0000b4008 10

// The zero value of a pointer is nil
var z *int
fmt.Println(z) // nil

// Dereferencing a nil pointer will result in a runtime panic
// fmt.Println(*z) // panic: runtime error: invalid memory address or nil pointer dereference

// new allocates memory for a type and returns a pointer to it
var a *int = new(int)
fmt.Println(a) // 0xc0000b4010
fmt.Println(*a) // 0

// To turn a constant into a pointer, use a helper function that takes a value
// and returns a pointer to it
func intPtr(i int) *int {
 return &i
}
var b *int = intPtr(10)

// A pointer is used to indicate that a parameter is mutable
func addOne(x *int) {
 *x++
}
addOne(b)
fmt.Println(*b) // 11

// To update the value of a pointer inside a function, dereference it and assign it a new value
func updatePointer(x *int) {
 *x = 2
}
updatePointer(b)
fmt.Println(*b) // 2

Types, Methods, and Interfaces

Types can be declared at any level, including at the package level.

Methods

Method names cannot be overloaded.

 go
 
 
 
type Person struct {
 FirstName string
 LastName string
 Age int
}

func (p Person) String() string {
// ^
// p is a receiver of the String method
 return fmt.Sprintf("%s %s, age %d", p.FirstName, p.LastName, p.Age)
}

func main() {
 p := Person{"Domi", "Noes", 42}
 fmt.Println(p.String()) // Domi Noes, age 42
}

Pointer Receivers and Value Receivers

Use a pointer receiver when the method needs to modify the receiver.
Use a pointer receiver when nil must be handled.
A value receiver is used when the method doesn’t need to modify the receiver.
If a type has a pointer receiver, all methods should have pointer receivers for consistency.
Usually, there’s no need for getters and setters when using structs: just use the fields directly.
Pointer receiver methods should check for nil values.

Methods are also functions

We can use a method value to turn a method into a function.
A method expression is used to turn a method into a function that takes the receiver as the first parameter.
They can be use for dependency injection.

 go
 
 
 
// Using a method value
func main() {
 p := Person{"Domi", "Noes", 42}
 f := p.String // using an instance of the struct
 fmt.Println(f()) // Domi Noes, age 42
}

// Using a method expression
func main() {
 p := Person{"Domi", "Noes", 42}
 f := Person.String // using the type itself
 fmt.Println(f(p)) // Domi Noes, age 42
}

Functions vs. methods

Use a function when there is no need to modify the receiver.
Use a method with a struct receiver when there is a need to modify data at runtime.

Type declarations are not inheritance

Declaring a type based on another type is not inheritance: there is no hierarchy.
Type conversion is used to convert a value from one type to another.

Types serve as executable documentation

They can be used to make code more readable and self-documenting.
A Percentage type can be used to make it clear what a value is instead of an int.

Use iota for enumeration (sparingly)

Go does not have an enumeration type.
iota is a built-in constant generator that starts at 0 and increments by 1 for each subsequent constant.
iota can be used to create a set of constants that are related to each other.
It should be used for “internal” purposes only, when the constants are referred to by name – not by value.
It is useful to differentiate between sets of values, not to rely explicitly on the values themselves.
If the first value in the constant block (with value 0) is not really initialized or the value 0 does not make sense, it can be named with an _ to skip 0.

 go
 
 
 
type Color int

const (
 Red Color = iota
 Green
 Blue
)

func main() {
 fmt.Println(Red, Green, Blue) // 0 1 2
}

Embedding for composition

Types can be embedded to encourage composition.

 go
 
 
 
type Person struct {
 FirstName string
 LastName string
 Age int
}

type Employee struct {
 Person // embedded field
 EmployeeID int
}

func main() {
 e := Employee{
 Person: Person{
 FirstName: "Domi",
 LastName: "Noes",
 Age: 42,
 },
 EmployeeID: 12345,
 }
 fmt.Println(e.FirstName, e.LastName, e.Age, e.EmployeeID) // Domi Noes 42 12345
}

Interfaces

Interfaces are declared as a type with the interface keyword.
The methods defined in an interface are the methods that a type must implement to be considered an implementation of that interface: this is referred to as the method set.
They can be declared in any block.
They usually end with er (e.g., Stringer, Reader, Writer).
They are implemented implicitly.
Go is a blend of duck typing (e.g., dynamic behavior in Python) and structural typing (Java interfaces).
Use built-in interfaces from the standard library as much as possible.
Just like structs, interfaces can be embedded.
Interfaces are the only abstract type in Go.

 go
 
 
 
// Switching between the logic providers is as simple as changing the type of the L field.
import "fmt"

type LogicProvider struct{}

type LogicProvider2 struct{}

func (lp LogicProvider2) Process(data string) string {
 fmt.Println("LogicProvider2: Process")
 return data
}

func (lp LogicProvider2) Rework(data string) string {
 return data
}

func (lp LogicProvider) Process(data string) string {
 return data
}

func (lp LogicProvider) Rework(data string) string {
 fmt.Println("LogicProvider1: Rework")
 return data
}

type Logic interface {
 Process(data string) string
 Rework(data string) string
}
type Client struct {
 L Logic
}

func (c Client) Program() {
 data := "data"
 c.L.Process(data)
 c.L.Rework(data)
}

func main() {
 c1 := Client{
 L: LogicProvider{},
 }
 c1.Program()

 c2 := Client{
 L: LogicProvider2{},
 }
 c2.Program()
}

// Output:
// LogicProvider1: Rework
// LogicProvider2: Process

Take interfaces and return structs

Returning interfaces increases coupling.
It is better to return structs and take interfaces as parameters.

Empty interfaces

If an interface is nil, invoking a method on it will result in a runtime panic.
An empty interface matches any type, because it requires implementing no methods.
They can be used when receiving data from an external source (e.g., a database or JSON) and the type is unknown.
Using interface{} is a code smell: it should be used sparingly.

Type assertions and type switches

Use the comma ok idiom to avoid a type assertion from panicking.
Type assertions are checked at runtime.
Type conversions are checked at compile time.
A type switch is used to check the type of an interface.
In the case of errors, use errors.Is and errors.As to test for specific errors.
Add a default case to switch statements to catch unexpected types.
It can be safer to keep interfaces unexported.

 go
 
 
 
func main() {
 var i interface{} = 42
 v1, ok1 := i.(int)
 fmt.Println(v1, ok1) // 42 true
 v2, ok2 := i.(string)
 fmt.Println(v2, ok2) // "" false

 // type switch
 switch v := i.(type) {
 case int:
 fmt.Println("int", v) // int 42
 case string:
 fmt.Println("string", v)
 case float64, float32: // check both at once
 fmt.Println("float", v)
 default:
 fmt.Println("unknown", v)
 }

 // type assertion using optional interface
 if s, ok := i.(MySpecificInterface); ok {
 // myInterface satisfies MySpecificInterface
 s.SpecificMethod()
 }
}

Function types are a bridge to interfaces

Functions can implement interfaces.
Go uses this to implement the http.Handler interface.
Small interfaces are encouraged.
If a function can depend on many other functions, it is better to use a struct with methods.

 go
 
 
 
type HandlerFunc func(http.ResponseWriter, *http.Request)

func (f HandlerFunc) ServeHTTP(w http.ResponseWriter, r *http.Request) {
 f(w, r)
}

func main() {
 http.Handle("/", HandlerFunc(func(w http.ResponseWriter, r *http.Request) {
 fmt.Fprintln(w, "Hello, world!")
 }))
 http.ListenAndServe(":8080", nil)
}

An actual, basic HTTP server:

 go
 
 
 
package main

import (
 "fmt"
 "net/http"
)

type HandlerFunc func(http.ResponseWriter, *http.Request)

func (f HandlerFunc) ServeHTTP(w http.ResponseWriter, r *http.Request) {
 f(w, r)
}

func main() {
 http.Handle("/", HandlerFunc(func(w http.ResponseWriter, r *http.Request) {
 p, err := fmt.Fprintln(w, "Hello, world!")
 if err != nil {
 return
 }
 fmt.Println(p)
 }))
 err := http.ListenAndServe(":8080", nil)
 if err != nil {
 return
 }
}

Implicit Interfaces Make Dependency Injection Easier

“[…] code should explicitly specify the functionality it needs to perform its task”.
If a type has methods that match an interface’s signature, it implicitly satisfies that interface.
Go’s implicit interfaces make dependency injection easier.
Interfaces are used to decouple code.
The client’s code should not be responsible for creating the dependencies it needs.
The client’s code defines the interfaces and can customize the method set it needs.

Here is an example of an implicit interface:

 go
 
 
 
package main

import "fmt"

type Talker interface {
 Say() string
}

// Dog implicitly implements the Talker interface
type Dog struct{}

func (d Dog) Say() string {
 return "Woof!"
}

func Speak(t Talker) {
 fmt.Println(t.Say())
}

func main() {
 d := Dog{}
 Speak(d) // Woof!
}

And an example demonstrating dependency injection:

 go
 
 
 
package main

import (
 "fmt"
)

// Logger is an implicit interface with a Log method
type Logger interface {
 Log(message string)
}

// SimpleLogger implicitly satisfies Logger by implementing Log
type SimpleLogger struct{}

func (sl SimpleLogger) Log(message string) {
 fmt.Println(message)
}

// AdvancedLogger implicitly satisfies Logger by implementing Log
type AdvancedLogger struct{}

func (al AdvancedLogger) Log(message string) {
 fmt.Println("ADVANCED: " + message)
}

// CompositeLogger combines SimpleLogger and AdvancedLogger
// Again, it implicitly satisfies Logger by implementing Log
type CompositeLogger struct {
 loggers []Logger
}

// Log messages with multiple loggers
func (cl CompositeLogger) Log(message string) {
 for _, logger := range cl.loggers {
 logger.Log("[CompositeLogger] " + message)
 }
}

// Greeter contains logic to greet
type Greeter struct {
 logger Logger // Logger is injected
}

// NewGreeter injects dependencies and returns a new Greeter
func NewGreeter(l Logger) Greeter {
 return Greeter{logger: l}
}

// Greet uses the Logger to log a greeting
func (g Greeter) Greet() {
 g.logger.Log("Hello, dependency injection and implicit interface!")
}

func main() {
 logger1 := SimpleLogger{} // Create Logger
 greeter1 := NewGreeter(logger1) // Inject Logger into Greeter
 greeter1.Greet() // Use Greeter
 // Logs:
 // Hello, dependency injection and implicit interface!

 logger2 := AdvancedLogger{} // Create Logger
 greeter2 := NewGreeter(logger2) // Inject Logger into Greeter
 greeter2.Greet() // Use Greeter
 // Logs:
 // ADVANCED: Hello, dependency injection and implicit interface!

 logger3 := CompositeLogger{loggers: []Logger{SimpleLogger{}, AdvancedLogger{}}}
 greeter3 := NewGreeter(logger3) // Inject CompositeLogger into Greeter
 greeter3.Greet() // Use Greeter
 // Logs:
 // [CompositeLogger] Hello, dependency injection and implicit interface!
 // ADVANCED: [CompositeLogger] Hello, dependency injection and implicit interface!
}

Generics

Generics Reduce Repetitive Code and Increase Type Safety

Generics are “type parameters” that allow one to write functions and data structures that can work with any type.
They reduce repetitive code and increase type safety.

Generic Functions Abstract Algorithms

Generic functions can be used to abstract algorithms.
They can be used to write functions that work with any type.

 go
 
 
 
package main

import "fmt"

// Max returns the maximum of two values
func Max[T any](a, b T) T {
 if a > b {
 return a
 }
 return b
}

Generics and interfaces

Generics can be used with interfaces.
They can be used to write functions that work with any type that satisfies an interface.

 go
 
 
 
package main

import "fmt"

// Stringer is an interface with a String method
type Stringer interface {
 String() string
}

// Print calls the String method on a value that satisfies the Stringer interface
func Print[T Stringer](s T) {
 fmt.Println(s.String())
}

Errors

Go does not have exceptions.
Errors are values.
They are the last return value of a function (by convention).
The error interface is defined in the standard library.
The errors package is used to create errors.
The fmt package is used to print errors.
The errors.Is function is used to check for specific errors.
The errors.As function is used to check for specific errors and get the underlying error.
The errors.Unwrap function is used to get the underlying error, but errors.Is and errors.As are more commonly used for this.
The errors.New function is used to create errors.
The errors.Errorf function is used to create errors with formatting.
When a function returns an error, it is expected that the caller will check for it.
If a function does not return an error, its value will be nil, because it is the zero value for interfaces.
Error messages should not be capitalized or end with punctuation, nor contain newline.
Because all values must be read, errors cannot be ignored implicitly.
Because the main code is unindented and the error handling code is indented, the code’s purpose is easier to follow.

Sentinel errors

Sentinel errors are errors that are predefined and can be checked for equality.
They are defined at the package level.
They start with Err (except for io.EOF).
They should be treated as read-only.
They are used to indicate it is not possible to continue processing (e.g., ErrFormat for ZIP files).
Whenever possible, use existing sentinel errors from the standard library.

Wrapping errors

To give additional context to an error, wrap it with fmt.Errorf.
A series of errors can be wrapped with fmt.Errorf and errors.Unwrap: these are called error chains.
If context is not required, a brand new error can be created with errors.New or errors.Errorf.
The %v verb can be used to print the error chain without wrapping the error (e.g., fmt.Errorf("internal failure: %v", err)).

`Is` and `As`

Use errors.Is to check for specific errors.
Use errors.As to check for specific errors and get the underlying error.
errors.As can take as the second parameter a pointer to a variable of the type of the error we are looking for, but just as well it can take a pointer to an interface.

Wrapping Errors with defer

Using named return values and defer can make error handling easier because the error can be formatted only once at the end of the function.

Panic and recover

A panic is used to indicate that the program cannot continue (e.g., out of memory error or trying to read beyond the end of a slice).
When a panic occurs, the program stops executing and the stack is unwound, running all deferred functions until the main function is reached.
panic and recover are not intended to be used for error handling.
It is better to explicitly handle errors than to use panic and recover because it is not clear when calling recover what failed exactly.

Getting a Stack Trace from an Error

Go doesn’t provide a stack trace by default outside of a panic state.
%+v can be used to print the stack trace with fmt.Printf.
Pass the -trimpath flag to go build to remove the absolute path from the stack trace, which otherwise shows full paths to files.

Modules, Packages, and Imports

The module is the root of the package tree. It can be defined in go.mod or inferred from the directory structure. For a GitHub repository, it is inferred from the URL as in module github.com/{USER}/{PROJECT}.
Keep a single module per repository.

`go.mod`

Use go mod init MODULE_PATH to create a new module (go.mod file).
The module path is case-sensitive.
The minimum version of Go required to build the module can be specified with go mod init MODULE_PATH GO_VERSION and it appears in the go.mod file below the module declaration.
There can be a require directive for each dependency.
There are also two optional sections: replace and exclude.
- replace is used to replace a dependency with a local version.
- exclude is used to exclude a dependency from the build.

 go
 
 
 
// Example of a go.mod file
module github.com/{USER}/{PROJECT}

go 1.xx

require (
 github.com/{OTHER_USER}/{OTHER_PROJECT} v0.0.0-20200921021027-5abc380940ae
 github.com/shopspring/decimal v1.2.0
)

Building packages

Imports and exports

Import statements allow accessing exported constants, variables, functions and types from another package.
An exported identifier starts with a capital letter. It cannot be accessed from another package without an import statement.

Creating and accessing packages

The first line of the file should be package {PACKAGE_NAME}. It’s a package clause.
Next is the import section.
- Importing from the standard library doesn’t require a path.
- Any other imports require a path, using the module path as a prefix and appending the path to the package.
- Not using any identifier from a path will result in a compiler error. Hence, all code included in the build will be used.
- It is best to always use absolute paths for clarity.
- The name of a package is determined by its package clause, not by the path being imported. In general, the package name should match the last element of the path.
- The main package cannot be imported as it is the entrypoint of the application.
Package names are in the file block: the package name is the same for all files in the same directory and must be present.

Naming packages

The package name should be descriptive.
Avoid util and common packages. Create more packages with fewer functions instead.
Don’t include the name of the package in functions, as this will be disambiguated by the package name when importing.

Organizing a module

There is no official structure.
The cmd directory is used for executables. There can be multiple executables produced by different applications from a module.
When there are a bunch of files at the root to manage deployment and testing, it is a good pattern to put all packages inside a pkg directory.
- Inside pkg, limit dependencies between packages by organizing the code according to the functionality it provides.
- A good primer on the topic is GopherCon 2018: Kat Zien - How Do You Structure Your Go Apps.

Overriding a package’s name

When names collide, you can use an alias to rename a package.
- In the standard library, both "crypto/rand" and "math/rand" are imported as "rand", but they can be disambiguated with an alias such as crand "crypto/rand".
Using the . alias is discouraged because it makes it harder to understand where a function is coming from as it will import all exported identifiers from the current namespace (same idea as import * from ... in Python).
Package names can be shadowed, which renders it inaccessible. Always resolve conflicts by using an alias instead.

Package comments and `godoc`

Comments must be placed right above the item to be documented.
Comments start with // and can be multiline.
A blank line with // is used to create paragraphs.
Preformatted text can be inserted with indentation.
Comments before the package clause create package-level documentation.
- If the package has a lot of documentation, it is better to put it in a separate file called doc.go.
Comments should start with the name of the item being documented.

Internal package

Use the internal directory to create packages that are only accessible from the parent and sibling packages.
Trying to access an internal package from outside the module will result in a compiler error.

The `init` function

If there is an init function in a package, it runs as soon as the package is referenced by another package.
It has not input or output parameters and can only cause side-effects within the package.
There can be multiple init functions in a package, even in the same file, although this setup is discouraged.
A blank import can be used to run the init function of a package without using any of its exported identifiers, e.g., import _ "github.com/lib/pq". Explicit is better than implicit: this is an obsolete pattern.

Circular dependencies

They are not allowed to keep the code readable and the compiler fast.
If two packages depend on each other, they should probably be merged into a single package.
If two packages with circular dependencies are still preferred, it may be possible to move only the culprits into a separate package so that both packages can import them.

API renaming and organizing

To avoid breaking a function name, a new method can be added with the new name, calling the old one.
Constants can be re-declared with the same types, but with a different name.
To rename exported types, we can create an alias such as type Bar = Foo. In this case, new methods should still be added to the original type to preserve backward compatibility.
Caution: a field name cannot be changed without breaking backward compatibility.

Working with modules

Importing third-party code

go.mod is used to manage dependencies.
go.mod is automatically populated when a go command runs and a dependency is required (e.g., go build, go list, go run, go test).
go.sum is used to verify the integrity of the dependencies.
god.mod and go.sum should both be committed to version control.

Working with versions

go list is used to list dependencies used by a project.
- By default, it lists packages used.
- The -m flag is used to list modules used.
- Appending the -versions flag to the previous command lists all versions of the dependencies.
go get can be used to downgrade or upgrade a dependency (e.g., go get github.com/{USER}/{PROJECT}@v1.0.0).
- It can also be used to add a new dependency.
- It can be used to remove a dependency by adding the -u flag.
- Changes will be reflected in go.mod and go.sum.
Dependencies might be shown with // indirect next to them. This means that the dependency is not used directly by the project, but by one of its dependencies.
- It can be added when the project uses that dependency directly with a newer version than what is declared in the dependency’s go.mod file.
Go follows semantic versioning (SemVer).
There is an import compatibility rule: all minor and patch versions should remain compatible and if not, this is considered a bug.
- Instead of importing multiple versions of the same library as with npm, Go will import the highest version of the library that satisfies the requirements.

Updating to compatible versions

Use go get -u=patch DEPENDENCY_PATH to update to the latest compatible patch version.
go get -u DEPENDENCY_PATH will update to the most recent compatible version.

Updating to incompatible versions

Go follows the semantic import versioning rule: for all major versions greater than 1, the major version is included in the import path, e.g., "github.com/{USER}/{PROJECT}/v2".
Once a new major version is used in the project, go build will update the go.mod file to use the new major version.
Older versions may still be present in go.mod: go mod tidy can be used to remove them.

Vendoring

go mod vendor is used to create a vendor directory with all the dependencies to ensure reproducible builds.
It dramatically increases the size of the project in version control.

`pkg.go.dev`

It automatically indexes Go projects.
It is used to search for packages and their documentation.
It publishes the godocs, license, README, the module’s dependencies and which other open source projects depend on it.

Publishing modules

Go favors permissive licenses (e.g., MIT, BSD, Apache).
There is no need for a central repository, as Go uses the module path to find the module.

Versioning modules

Minor and patch versions should be compatible and are easy to manage.
Major versions are slightly more difficult to manage. For instance, let’s go from v1 to v2.
- Create a directory to put all the old code in, named v2, including README and LICENSE files.
- Create a branch.
  - Name the branch v1 if the old code goes in it.
  - Name the branch v2 if the new code goes in it.
- Make sure the module path in go.mod ends with /v2.
- Update all import paths to use the new module path.
- Create a tag for the new version.
  - Name the tag v2.0.0.
  - Tag the main branch if the new code goes in it.
  - Otherwise, tag the v2 branch.
If breaking changes might be introduced while on the new version, use a pre-release version, e.g., v2.0.0-alpha.1.
The open source project mod can be used to automate this process.
The Go Blog has a post on the topic.

Module proxy servers

Google manages a module proxy server that fetches all versions of all publicly available modules.
Google also maintains a sum database, which stores the checksums of all the modules.
Modules are only installed from the proxy server if they are not already present in the local cache and if the checksums match.

Specifying a module proxy server

The GOPROXY environment variable can be used to specify a module proxy server. To use GoCenter, set it to GOPROXY=https://gocenter.io.
If the GOPROXY environment variable is set to direct, the module proxy server will not be used and the module will be downloaded directly from the source.
Projects such as athens can be used to create a local module proxy server.

Concurrency in Go

When to use concurrency

Concurrency is not parallelism.
Concurrency is useful when there are multiple tasks that can be executed independently.
Concurrency brings benefits when a process takes a long time to complete.
Read The Art of Concurrency for more information.
The book Concurrency in Go is also a great resource.

Goroutines

“Goroutines are lightweight processes managed by the Go runtime”.
Because Go manages goroutines, they are cheap to create and destroy (no need to create system-level resources).
They are memory efficient because they are allocated on the stack with small initial sizes.
Switching between goroutines is fast because it is managed by the Go runtime within a process.
Go optimizes how work is distributed across goroutines.
- For more details on this, watch GopherCon 2018: Kavya Joshi - The Scheduler Saga.
A goroutine starts by calling a function with the go keyword in front of it.

Channels

Channels are used to communicate between goroutines.
They are a built-in type that require the make function to create them, e.g., ch := make(chan int).
The zero value of a channel is nil.
They are passed as parameters to functions as a pointer.

Reading, writing, buffering

The <- operator is used to send and receive data from a channel. It indicates the direction of the data flow.
A function parameter can specify the direction of the channel, e.g., func f(ch <-chan int) will make it so that the channel can only be read from. Likewise, func f(ch chan<- int) will make it so that the channel can only be written to.
By default, channels are unbuffered, meaning that they can only hold one value at a time. They should be used most of the time.
A channel can be buffered by specifying the buffer size when creating it. E.g., ch := make(chan int, 10).
len(ch) is used to get the number of elements in the channel.
cap(ch) is used to get the capacity of the channel.
The capacity of a channel cannot be changed after it is created.

`for-range` and channels

for-range can be used to read from a channel until it is closed, or until a break or return statement is encountered.
There is a single variable declared for the channel, which is the value read from the channel.

 go
 
for v:= range ch {
 fmt.Println(v)
}

Closing a channel

Close a channel with close(ch).
Writing to a closed channel will result in a runtime panic.
Attempting to read from a closed unbuffered channel will return the zero value of the channel’s type.
Reading from a closed buffered channel will return the remaining values in the channel until it is empty, then it will return the zero value of the channel’s type.
To know if a channel is closed, use the second return value of the receive operation with the comma ok idiom, e.g., v, ok := <-ch.
- ok will be true if the channel is open and false if it is closed.

How channels behave

They pause the execution of the goroutine until a value is available to read from the channel.
They pause the execution of the goroutine until a value can be written to the channel.
A panic will occur if a value is written to a closed channel or when trying to close a closed channel or a nil channel.
Make the writer responsible for closing the channel.

`select`

Starvation is when a goroutine is waiting for a resource that is never available.
It looks very similar to a switch statement.
A case in a select statement is executed when the channel is ready to be read from or written to.
If multiple case statements are ready, one is chosen at random – with switch, the first match is always chosen. This solves the starvation problem since all cases are checked at once.
select also deals with deadlock issues: if all channels are blocked, it will execute the default case.
select is often used in a loop to keep reading from a channel until it is closed.
- Having a default case inside a loop for a select is most certainly not what is intended as it will run constantly.

Concurrency practices and patterns

Keep APIs concurrency-free

Never expose channels or mutexes in an API. If a channel is exposed, the user will have to manage it, know whether it is buffered or not, closed or not or nil. The user could also trigger deadlocks.

Goroutines, for loops, and varying variables

Instead of shadowing variables in a for loop, pass them as parameters to the goroutine.

 go
 
 
 
// Don't do this
for _, v := range a {
 v := v // shadowing
 go func() {
 ch <- v * 2
 }()
}

// Do this instead
for _, v := range a {
 go func(val int) { // value captured by the closure
 ch <- val * 2
 }(v)
}

Always clean up your goroutines

If a goroutine is not cleaned up, it will keep running until the program exits: this is called a “goroutine leak”.

“Done channel pattern”

If multiple goroutines are running, it is useful to have a way to signal them to stop.
A channel (named done) can be used to signal the goroutines to stop.

Using a cancel function to terminate a goroutine

A cancel function can be used to terminate a goroutine.
It is a function that performs a cleanup and closes a channel.

When to use buffered and unbuffered channels

Buffered channels are useful when limiting the number of goroutines that can access a resource at the same time, when limiting the amount of work that gets queued up or when the number of goroutines is known.
Buffered channels are good to gather results from multiple goroutines when a deterministic order is not required.

Backpressure

Backpressure is a way to signal to a goroutine that it should slow down.
It is implemented with buffered channels.

Turning off a case in a `select`

Setting a variable’s channel to nil will turn off the case in a select statement.

 go
 
 
 
for {
 select {
 case v, ok := <-in:
 if !ok {
 in = nil // kills the case
 continue
 }
 fmt.Println(v)
 // ... other cases ...
 case <-done:
 return
 }
}

Time out code

Use time.After to time out code.

 go
 
 
 
select {
// this requires context cancellation to free up resources
// if the function is not finished before the timeout,
// else it will keep processing in the background
case v := <-ch:
 fmt.Println(v)
case <-time.After(1 * time.Second):
 fmt.Println("timed out")
}

Using `WaitGroups`

The done channel pattern works well when waiting for a single goroutine to finish.
When waiting for multiple goroutines to finish, use a sync.WaitGroup.
Add is used to add a goroutine to the wait group.
Done is used to signal that a goroutine is done.
Wait is used to wait for all goroutines to finish.
Use them when some cleanup is required after all goroutines are done.
ErrGroup can be used to wait for multiple goroutines to finish and return an error if one of them fails.

 go
 
 
 
func main() {
 var wg sync.WaitGroup
 var a = []int{1, 2, 3, 4, 5}
 wg.Add(len(a))
 ch := make(chan int, len(a))
 for _, v := range a {
 // The wait group is passed with a closure
 // Otherwise it would need to be passed as a pointer
 // So all goroutines share the same instance
 go func(val int) {
 defer wg.Done() // called even if panic
 ch <- val * 2
 }(v)
 }
 wg.Wait()
 close(ch)
 for v := range ch {
 fmt.Println(v)
 }
}

Running code exactly once

once.Do is used to run code exactly once.
It can be found in the sync package.
It needn’t be initialized, as the zero value is usable.
It is best to use the minimum amount of concurrency possible.

 go
 
 
 
var once sync.Once

func main() {
 once.Do(func() {
 fmt.Println("Only once")
 })
 once.Do(func() {
 fmt.Println("Only once") // will not be printed
 })
}

When to use mutexes instead of channels

Mutex stands for mutual exclusion.
Mutexes limit access to a resource to a single goroutine at a time with a locking mechanism (Lock and Unlock, which must be used carefully to avoid creating deadlocks, especially in functions implemented recursively).
They require to do more bookkeeping than channels.
They should never be copied, just like sync.WaitGroup and sync.Once.
Use mutexes when there is a shared resource that needs to be protected, such as a field in a struct.
RWMutex is used when there are multiple readers and a single writer. The critical section is protected by a write lock, while the read lock is used to read the resource by multiple goroutines.
Sometimes, performance issues with channels can be solved by using mutexes instead.

Atomics

Atomics are used to perform atomic operations on integers and pointers.
They are more niche than mutexes and channels and as such, they are not covered in this introductory summary on Go.

The Standard Library

It is battery-included, just like Python.

`io` and Friends

io is used to read (io.Reader) and write (io.Writer) data.
io.Closer is used to close a resource.
Read (io.ReaderAt) and write (io.WriterAt) data at a specific offset.
io.Seeker is used to seek to a specific offset.
There are other combinations to define more explicitly the code’s intent (ReadWriter, ReadCloser, WriteCloser, ReadWriteCloser, ReadSeeker, WriteSeeker, ReadWriteSeeker).
The io package is a great example of the power of interfaces through simple abstractions.
Reader, Writer, and Scanner from the bufio package are used to read and write data more efficiently on larger datasets.

`time`

time is used to work with dates and times.
time.Time is used to represent a date and time. It includes the usual constants for days of the week, months, etc.
time.Duration is used to represent a duration.
time.Parse is used to parse a string into a time.Time value.
time.Format is used to format a time.Time value into a string.
time.Now is used to get the current time.
time.Sleep is used to pause the execution of a goroutine for a specified duration.
time.After is used to create a channel that will receive a value after a specified duration.
time.Tick is used to create a channel that will receive a value at regular intervals.
time.Timer is used to create a timer that will send a value on a channel after a specified duration.
time.Ticker is used to create a ticker that will send a value on a channel at regular intervals.
time.NewTicker is used to create a new ticker.
time.NewTimer is used to create a new timer.
time.Since is used to get the time elapsed since a specified time.
time.Until is used to get the time until a specified time.
time.AfterFunc is used to execute a function after a specified duration.
Use Equal to compare two time.Time values.

Monotonic time

Go uses a monotonic clock to measure time.
A monotonic clock is a clock that counts up from the start of the computer.

Timers and timeouts

Use time.NewTicker to create a ticker that will send a value on a channel at regular intervals.
Use time.NewTimer instead of time.Tick, as it can be stopped and reset.

Encoding and JSON

“Marshalling” is the process of converting a data structure into a byte stream.
“Unmarshalling” is the process of converting a byte stream into a data structure.

Use struct tags to add metadata

Struct tags are strings written inside backticks that can be added to struct fields to add metadata.
They can span only one line, taking the format `tagName:"tagValue"`.
go vet can be used to check for struct tags validity.
If no struct tag is specified, the field name will be used instead.
Use a - (dash) for a field name to ignore it.
,omitempty can be added right after the field name to omit a field if it is empty.
While annotations can make the code more declarative and short, they can also make it harder to read and understand. Go for readability first.

Unmarshalling and marshalling

json.Unmarshal is used to unmarshal a JSON byte stream into a data structure.
json.Marshal is used to marshal a data structure into a JSON byte stream.

JSON, readers, and writers

json.Decoder is used to decode a JSON byte stream into a data structure.
json.Encoder is used to encode a data structure into a JSON byte stream (e.g., encoding a file given an interface: json.NewEncoder(tmpFile).Encode(toFile)).
json.NewDecoder is used to create a new decoder (e.g., decoding a file given an interface: json.NewDecoder(tmpFile2).Decode(&fromFile)).
json.NewEncoder is used to create a new encoder.

Encoding and Decoding JSON Streams

json.Decoder and json.Encoder can be used to encode and decode JSON streams.

Custom JSON Parsing

json.Unmarshaler is used to implement custom JSON parsing.
json.Marshaler is used to implement custom JSON marshalling.

`net/http`

As a modern language, Go has a built-in HTTP client and server.
Third-party libraries of interest in this space include chi and Gorilla Mux for routing needs and alice to deal with middleware chaining.

Client

Don’t use the default client in production.
Don’t use the functions to make HTTP requests directly in production as they don’t have timeouts.

Server

http.ListenAndServe is used to start a server.
The middleware pattern is used to add functionality to a server, so as to check for authentication, logging, etc.

The Context

Context is not a new feature: it is an instance that meets the context.Context interface and gets passed around as the first argument to functions, which is usually named ctx.
context.TODO is used when a context is required but there is no context available. It shouldn’t be used in production.

Cancellation

context.WithCancel is used to create a context that can be cancelled.
When a cancellable context is created, a cancel function is returned. It is used to cancel the context and it must be called (at least once) using defer.

Timers

A server can do a few things to manage its load:

It can limit the number of concurrent requests it accepts.
- It can be done by limiting the number of goroutines.
It can limit the number of requests queued up.
- This can be handled with a buffered channel.
Limit the amount of time a request can take.
- The context can be used to do this.
- context.WithTimeout is used to create a context that will be cancelled after a specified duration.
- context.WithDeadline is used to create a context that will be cancelled at a specified time.
Limit the resources a request can use (memory, disk space…).
- There is no built-in solution for this in Go.

Values

context.WithValue is used to create a context with a value.

Writing Tests

Basics of testing

The testing package is used to write tests.
go test is used to run tests and generate reports.
Tests are located in files with the suffix _test.go in the same package as the code being tested, so they have access to unexported identifiers.
Test functions start with Test and take a *testing.T parameter. They do not return any value.
The test function name should be descriptive. It can start with Test_ to indicate the function under test is unexported.

Reporting failure

Use t.Error or t.Errorf to report a failure and continue the test. Use it to conveniently report as many failures as possible.
Use t.Fatal or t.Fatalf to report a failure and stop the test. Use it when subsequent tests on the same function will fail or trigger a panic.
Use t.Log to log information about the test.

Setting up and tearing down

TestMain is used to set up and tear down tests.
- It is called once before and after all tests, not between each test.
- Can be used to set up and tear down a database, for instance.
- It can be used when package-level variables need to be initialized, although this probably means the code needs refactoring.
- It takes a *testing.M parameter. Setup can be done at the beginning, then exitVal := m.Run() is called to run tests, then teardown can be done at the end, returning the exit value with os.Exit(exitVal).
Individual test functions receive a *testing.T parameter, which has a Cleanup method that can be used to clean up after a test.
- The Cleanup method is similar to the defer statement but can be useful is the cleanup actions are performed for multiple tests from a helper function.

Storing sample test data

A directory named testdata can be created to store sample test data for the package under test, as the package directory is used for the currently working directory.
Each package accesses its own testdata directory, so it is possible to have multiple testdata directories in a project: up to one per package.

Caching test results

go test caches test results to speed up subsequent runs.
Tests recompile if the source code or the data in testdata changes.
go test -count=1 can be used to disable caching.

Testing your public API

Create a directory packagename_test to test the public API of a package, to be found at the same directory level as packagename.
Test files in this case will be named packagename_public_test.go.
Test your public API, not your implementation.
This is used to test the exported functions and methods, not the unexported ones.

Use go-cmp to compare test results

It will output the differences between the expected and actual values in a human-readable format.
It can be used to compare for strict equality as well as with custom comparators.

Table tests

Table tests are used to test a function with multiple inputs and outputs.
Table-driven tests are idiomatic in Go.
A slice of an anonymous struct is used to store the test cases.
The test function iterates over the test cases and runs the test for each of them.

Given the following:

 go
 
 
 
package adder

import (
 "testing"
)

func Add(a, b int) int {
 return a + b
}

func TestAdd(t *testing.T) {
 // Table of test cases
 tests := []struct {
 name string
 a, b int
 expected int
 }{
 {"Positive integers", 1, 2, 3},
 {"Negative integers", -1, -1, -2},
 {"Two zeros", 0, 0, 0},
 {"Negative and positive", -1, 1, 0},
 {"Two large integers", 100, 200, 300},
 }

 for _, test := range tests {
 t.Run(test.name, func(t *testing.T) {
 result := Add(test.a, test.b)
 if result != test.expected {
 t.Errorf("Add(%d, %d) = %d; want %d", test.a, test.b, result, test.expected)
 }
 })
 }
}

The output would look like this:

 
 
 
=== RUN TestAdd
=== RUN TestAdd/Positive_integers
--- PASS: TestAdd/Positive_integers (0.00s)
=== RUN TestAdd/Negative_integers
--- PASS: TestAdd/Negative_integers (0.00s)
=== RUN TestAdd/Two_zeros
--- PASS: TestAdd/Two_zeros (0.00s)
=== RUN TestAdd/Negative_and_positive
--- PASS: TestAdd/Negative_and_positive (0.00s)
=== RUN TestAdd/Two_large_integers
--- PASS: TestAdd/Two_large_integers (0.00s)
--- PASS: TestAdd (0.00s)
PASS
ok github.com/{USER}/{PROJECT}/adder 0.368s

Checking code coverage

The -cover flag can be used to check code coverage.
go test -coverprofile=coverage.out is used to generate a coverage profile.
Go ships with a very cool tool to visualize the coverage profile: go tool cover -html=coverage.out.

Benchmarks

The built-in testing framework can be used to write benchmarks.
Benchmarks are functions that start with Benchmark and take a *testing.B parameter.
See Profiling Go programs with pprof for more information on profiling Go programs.

Stubs

We can using function types and interfaces to create stubs.
A test function can implement an interface and be passed to the function under test.
A stub can be defined as a struct that implements the interface and has a field for each method of the interface. E.g., type MathSolverStub struct {}.
When testing larger interfaces, one can define a stub that implements only the methods required for the test.
For mocks, use a mocking library such as gomock or testify.

`httptest`

httptest is used to test HTTP servers without having to start them.
A complete, real-world example is provided in the test_examples repo.

Integration tests and build tags

Integration tests are used to test the interaction between multiple components.
A build tag is a comment that starts with // +build and is followed by a tag name, found on the first line of a file.
Files with no build tags are included in all builds.
If a file has a build tag like // +build integration, then it can be run with go test -tags=integration -v ./....
To skip tests that take a long time to run, use t.Skip.

 go
 
 
 
func TestFileLen(t *testing.T) {
 if testing.Short() {
 b.Skip("skipping test in short mode.")
 }
 // ...
}

// Skip it when testing in short mode
// go test -short -v ./...

Finding concurrency problems with the race checker

A data race is still possible in Go with its built-in concurrency features if a lock hasn’t been acquired.
Go comes with a race checker for just these cases: go test -race.
Adding “sleep” statements is definitely not the correct approach.
The race checker can also run after building a binary: go build -race. This is useful to detect race condition issues for code that is not covered by tests.
Note that the race checker makes the code about 10 times slower, so use it only when needed.

Here There Be Dragons: Reflect, Unsafe, and Cgo

Theses features are not used that often, but they are useful to know about.
You cannot make make methods with reflection.
It should only be used when there is no other way to do it.
It may increase maintenance cost, because crashes can happen in production due to the lack of type safety (Java cough Script cough).
This summary will only cover their starting point.

Reflection to work with types at runtime

This can be used to work with data that didn’t exist at compile time.
Use cases:
- Reading and writing from a database;
- Template engines;
- fmt uses it heavily;
- errors uses it to implement errors.Is and errors.As;
- sort uses it to sort slices of arbitrary types;
- Marshalling/unmarshalling JSON and XML;
- Comparing maps or slices for deep equality with reflect.DeepEqual.

Types, kinds, and values

A type is a description of a value’s structure and behavior.
A kind is a description of a type’s behavior.
A value is a representation of a type’s behavior.
A type can have multiple kinds.
A value can have multiple types.
reflect.TypeOf is used to get the type of a value.
reflect.Type is used to represent a type.

Making new values

reflect.New is used to create a new value of a type (reflect.Type as input, reflect.Value as output).

Use reflection to check if an interface’s value is `nil`

IsValid is used to check if a value is valid (e.g., iv := reflect.ValueOf(i), iv.IsValid(), iv.IsNil()).

Use reflection to write a data marshaler

reflect.ValueOf is used to get the value of a field.
reflect.Type is used to get the type of a field.
reflect.StructField is used to get the field’s metadata.
reflect.StructTag is used to get the struct tag.
There is a complete example of a CSV data marshaler on the Go Playground.

Build functions with reflection to automate repetitive tasks

reflect.MakeFunc is used to create a function.
reflect.ValueOf is used to get the value of a function.
reflect.TypeOf is used to get the type of a function.
Reflection makes the program slower.

You can build structs with reflection, but don’t

reflect.StructOf is used to create a struct.

`unsafe`… is unsafe

It allows manipulating memory directly.
Sizeof is used to get the size of a type (“returns how many bytes it uses”).
Offsetof is used to get the offset of a field (“returns the number of bytes from the start of the struct to the start of the field”).
Alignof is used to get the alignment of a field (“returns the byte alignment it requires”).
unsafe.Pointer is used to convert a pointer to a pointer of a different type. Pointer arithmetic is possible just like in C or C++.

Use unsafe to convert external binary data

unsafe is used to convert external binary data.
It can be use to gain performance when interacting with the system.
It can speed up marshalling and unmarshalling (about twice as fast for simple structs).

unsafe strings and slices

reflect.StringHeader is used to get the header of a string.

`cgo` is for integration, not performance

It is best used to integrate with C libraries.
cgo is the FFI (foreign function interface) of Go.
You can call C functions from Go… and even Go functions from C!
Garbage collection makes it hard to use cgo for performance.
Only use it when there’s no suitable Go library available.

A Look at the Future: Generics in Go

Go doesn’t convert types implicitly.
For a gentle introduction to the topic, there is Tutorial: Getting started with generics.

Generics reduce repetitive code and increase type safety

Generics are akin to “type parameters”.
Without generics, Go has to use interface{} and type assertions.
any is used to represent any type.
Generics allow specifying the type of a generic function’s parameters, such as func Sum[T any](a, b T) T { return a + b }.

Use type lists to specify operators

A “type list” is a list of types.

 go
 
 
 
type BuiltInOrdered interface {
 type string, int, int8, int16, int32, int64, float32, float64,
 uint, uint8, uint16, uint32, uint64, uintptr
}

Salient takeaways

Go is a practical language, valuing clarity of intent and readability (e.g., standard formatting is mandatory). It takes the best of other languages and leaves out the rest.
Comprehensibility and explicitness is more important than conciseness in idiomatic Go.
Go is “call by value”, meaning it makes copies of function parameters before passing them along.
Deployment is a breeze: a single binary file.
Go doesn’t have classes nor inheritance, but it has structs and interfaces.
Professionals use error handling profusely to make their programs more robust.

Conclusion

This book achieves its goal of teaching readers how to write idiomatic Go code that leverages the strengths of the language. It focuses on real-world examples and best practices to structure Go code, rather than just explaining language syntax. The book covers a wide range of topics including primitive types, control structures, composite types like arrays and maps, concurrency, reflection, testing, and more.

We covered the key takeaways from each chapter, providing a broad overview of important concepts. While not a replacement for reading the book in its entirety, this summary may serve as a helpful reference guide on the subject. Whether you are just starting with Go or are looking to improve your skills, Learning Go is an invaluable resource and I highly recommend it!

Resources and references

Articles

Evaluating code complexity triggers, use of complexity measures and the influence of code complexity on maintenance time, Springer
Go Modules: v2 and Beyond, go.dev
Go Release History, go.dev
Marshalling, Wikipedia
Profiling Go programs with pprof, Julia Evans

Books

Concurrency in Go, Katherine Cox-Buday, O’Reilly
Learning Go, Jon Bodner, O’Reilly
The Art of Concurrency, Clay Breshears, O’Reilly
The Go Programming Language, Alan A. A. Donovan and Brian W. Kernighan, Addison-Wesley Professional

Documentation

atof.go, cs.opensource.google
builtin documentation, pkg.go.dev
CodeReviewComments, GitHub
Effective Go, go.dev
ErrGroup, pkg.go.dev
Go By Example, gobyexample.com
Go Wiki, GitHub
How to Write Go Code, go.dev
Standard library documentation, pkg.go.dev
The Go Programming Language Specification, go.dev

Other open source projects referenced

Alice, GitHub
Chi, GitHub
go-cmp, GitHub
Gorilla Mux, GitHub
Learning Go: Code examples, GitHub
Mod, GitHub

Testing

gomock, GitHub
Profiling Go programs with pprof, Julia Evans
test_examples repo, GitHub
testing package, pkg.go.dev
testify, GitHub

Videos

GopherCon 2018: Kat Zien - How Do You Structure Your Go Apps, YouTube
GopherCon 2018: Kavya Joshi - The Scheduler Saga, YouTube

Book summary: Refactoring UI

Sébastien Lavoie — Sat, 09 Sep 2023 17:56:00 -0600

Introduction

It covers core principles like establishing consistent systems for recurring elements such as spacing and typography.
It advises generous use of white space and thoughtful visual hierarchy.
Steps for adding depth through lighting, shadows, and layers are outlined.
Recommendations are provided for working with images and illustrations effectively.
Tips on elevating default elements and breaking conventions to avoid typical, cookie-cutter designs are included.
Studying other designers’ work and unintuitive decisions is encouraged to expand one’s skills.
The goal is to equip readers with knowledge to thoughtfully craft interfaces that appear purposefully designed without simply mimicking common graphical tropes.
By learning fundamental visual design concepts tailored to user interfaces, anyone can create clean, functional, and visually engaging digital experiences.

Starting from scratch

Start with a feature, not a layout
- Design the shell and navigation later once you know what features you need.
Detail comes later
- Design with a big sharpie.
- Design in grayscale to use spacing, contrast and size to do the heavy lifting.
Don’t over-invest
- Go with low-fidelity sketches and wireframes first to move fast. Don’t get bogged down with details like fonts and colors early on.
Don’t design too much
- Work in cycles: design a feature simply, build it, fix errors.
- Be a pessimist: expect it to be hard to build, and postpone building it (e.g., commenting system without attachments at first).

Choose a personality

Font choice
- Elegant/classic: serif
- Playful: Sans-serif
- Plain: neutral sans-serif
Color
- Blue: safe and familiar, good overall pick.
- Gold: expensive, sophisticated.
- Pink: more fun, less serious.
Border radius
- Small – neutral, not much personality.
- Large – more playful.
- None – more serious/formal.
- Stay consistent – either rounded or not, same everywhere.
Language
- Choose the right words for the desired tone.
Decide what you actually want
- Look at the websites your expected customers use? Serious, then that’s the feel. Playful? Then that.

Limit your choices

Define systems in advance
- Have a working palette color and limit yourself to it.
- Use a type scale to avoid picking specific font sizes.
Designing by process of elimination.
Systematize as much as possible to avoid repetitive low-level decisions.

Hierarchy

Visual hierarchy refers to how elements appear in importance relative to one another.
Don’t let HTML semantics dictate visual styling – style based on visual hierarchy.
Size isn’t everything. Bold font communicates importance. De-emphasize secondary/tertiary information to highlight the most important elements.
Rely on font weight and color, not just font size, to create hierarchy.
Limit yourself to 2-3 colors and font weights to define hierarchy.
On colored backgrounds, pick a color with similar hue but lower saturation/lightness rather than just making text gray.
Avoid label: value formats – use formats, context, and combined labels/values for clarity without labels.
Treat labels as secondary content when needed – smaller, lower contrast, lighter weight.
De-emphasize competing elements instead of just emphasizing the key element.
Combine labels and values: e.g., instead of “In stock: 12”, use “12 left in stock”.
Treat labels as supporting content.
Design buttons based on hierarchy, not just semantics. Primary/destructive actions don’t always need prominence.

Layout and spacing

Start with too much white space, then remove until happy – often ends up being “just enough”.
Compact layouts have their place but should be a deliberate choice, not the default.
Establish a spacing and sizing system
- A linear scale won’t work
- Create a constrained spacing/sizing system based on a base unit (like 16px) to work faster with more consistency.
Don’t feel the need to fill the whole screen – give elements only the space they need.
Grids are overrated
- Not all elements should be fluid
- Don’t shrink an element until you need to.
- Grids can bring order but don’t have to dictate every layout decision.
Balance layouts in columns instead of just making things wider.
Relative sizing doesn’t scale.
- Fix element widths when flexibility isn’t needed.
- Scale elements independently – don’t rely on relative units like em.
Avoid ambiguous spacing: increase spacing between groups and reduce spacing within groups to show relationships.

Text

Establish a constrained type scale to speed up font size decisions and add consistency. Avoid strict modular scales.
Use pixel or rem units – not em – to guarantee sizes match the scale.
For UI design, pick scale values by hand instead of using a mathematical ratio.
Use high quality fonts – safe bets are neutral sans-serifs and system fonts.
Use the system stack if not sure: -apple-system, Segoe UI, Roboto, Noto Sans, Ubuntu, Cantarell, Helvetica Neue;
Favor fonts with 5+ weights and optimized for legibility.
Keep line length between 45-75 characters (20-35 em) for optimal readability.
Align mixed font sizes by their baselines, not centers.
Use taller line-heights for small text and shorter line-heights for large text.
Subtly emphasize links in UIs, no need for high contrast colors.
Left align most text, right align numbers in tables. Justify only with hyphenation.
Don’t center long form text.
Tighten letter-spacing for headlines set in text faces. Loosen letter-spacing for all-caps.
Not all links need to be emphasized boldly, especially in UIs where there is plenty of interactivity.

Colors

HSL (hue, saturation, lightness) is more intuitive to the human eye: use it instead of HEX and RGB.
- Hue is the color position on the color wheel. 0° is red, 120° is green, and 240° is blue.
- Saturation represents the vividness. 0% is grey while 100% is vibrant and intense.
- Lightness measures how close a closer is to black (0%) or white (100%), where 50% is the pure color.
HSL is not the same as HSB (lightness != brightness). In HSB, 0% brightness is always black but 100% brightness is white when saturation is set to 0%.
- With 100% saturation in both HSL and HSB, 100% brightness is equivalent to 50% lightness (pure color).
Don’t rely solely on color to convey information - support it with other indicators.

A good color palette

You need more colors than you think - aim for 10+ shades of greys, primary colors, and accent colors.
Grey
- There is a lot of grey in UIs: between 8-10 shades of grey is a good array of options to pick from. Use very dark grey instead of true black so it looks more natural.
Primary colors
- 1 or 2 is great, with 5-10 different shades (e.g., Facebook is basically blue). A very light shade is good for a tinted background (e.g., alerts), darker shades work great for text.
Accent colors
- Use sparingly.
- Needed to bring semantic meaning, such as red for danger, yellow for warnings and green for positive trends.
- For a complex UI, can be as many as 10 different colors, each with 5-10 shades.

Building a color palette

Define color shades upfront instead of generating on the fly. Pick base, darkest, lightest, then fill in gaps.
Don’t rely purely on math to generate shades - trust your eyes and tweak if needed.
Choose the base color first. This is the color in the middle that light and dark shades are derived from.
Center: pick a shade that would work well as a button background.
Extremes: pick the lightness color to work as a background and the darkest to work on top as text to get enough contrast.
Repeat the process to pick the remaining colors in between. 9 shades is a good sweet spot as it divides cleanly.
For grays, the base color doesn’t matter nearly as much: same process.

Perceived brightness

In the HSL system, the further away a color is from 50% lightness, the more saturated it should be so it doesn’t look washed out.
The perceived brightness of a color to the human eye can be calculated as follows:

brightness = √(0.299·r² + 0.587·g² + 0.114·b²) / 255

Green is perceived as more bright than red, which in turn is perceived as more bright than blue. With this data point, we can tweak the perceived brightness of a color by rotating the color wheel towards the desired perceived brightness.
Grays aren’t usually totally desaturated.
- To make them feel “cool” saturate them with some blue.
- To make them feel “warm”, saturate them with yellow or orange.
- The same “saturation curve” applies to grays: the further away from 50% brightness, the more they should be saturated to avoid looking washed out.

A word on accessibility

Instead of having elements like dark-colored pills with a white foreground color, flip the contrast by having light-colored pills with a dark version of the color in the foreground so they won’t grab too much attention.
Rotate the hue to make the color pop more. For instance, instead of cranking up the lightness to meet ideal contrast ratios, rotate the hue and possibly keep the saturation high for a vibrant contrast.
Don’t rely on color alone
- Add visual cues (like arrows for upward/downward trends). Designing in grayscale first will help spot these kinds of issues.
- For things like a graph, playing with contrast will produce clearer results.

Depth

Mimic how light works in the real world.
- Emulate a light source coming from above to make elements appear raised or inset.
- For instance, with a button, if light comes from above, then the top should be lighter (e.g., box-shadow inset) than the bottom (box-shadow, casting shadow only below).
- For elements that are “inset” (I.e., more depth in the center), the opposite effect should happen: more light reaches the bottom part (vertical box-shadow, going up) than the top part (box-shadow inset such that it casts some shadow towards the bottom).
Don’t overdo it.

Shadows

Use shadows to convey elevation
- When something appears to be raised from the background (e.g., bigger drop-shadow), it attracts the user’s focus.
- Small shadows are useful to attract attention to the primary action to be performed.
- Medium shadows are useful for things like a dropdown so it looks separate from the background.
- Large shadows are great for modals.
- 5 options is usually plenty.
- Can be used for interactivity, like when dragging a row from a table.
- A button can feel like it’s being “pressed” by reducing its shadow on click (i.e., it gets closer to the page or farther away from the user).
- Casting shadows
  - To simulate a shadow cast by direct light (e.g., a long shadow behind a plant getting some sunlight), use a larger, more diffuse and subtle shadow.
  - To simulate a shadow cast by ambient light (e.g., the shadow right below the plant’s pot where light can’t hardly reach), use a narrow and darker shadow.
  - Apply both types of shadow casting at once to create visually compelling effects.
Define a fixed set of shadows as an elevation system for consistency.
Even flat designs can have depth
- Flat design does not convey depth with shadows and gradients.
- Create depth with color: lighter colors feel closer.
- Can use short, offset shadows with no blur at all.
Overlap elements to create layers
- E.g., position a block with a negative margin so it appears on top of two different sections.
- For smaller elements like avatars, an invisible border around it can make it pop better.

Images and contrast

Images
- Use high quality photos - hire a pro or use stock sites for generics. Don’t expect good results from phone pics.
Text needs consistent contrast
- Add an overlay to background images to get readable headlines.
- Lower the image contrast. Adjust the brightness to compensate.
- Colorize the image with a single color.
- Use text shadows instead of overlays to preserve image dynamics.

Scale

Don’t scale up icons
Even for SVGs, they will lack detail if they were designed correctly at a smaller scale: use a more detailed icon instead.
If updating the icon is not an option, enclose it inside another shape (e.g., a circle with a background color) to occupy more space.
Don’t scale down screenshots
Details will be distorted and hard to read.
Either use a presentation from a smaller device (e.g., the tablet version on a desktop) or take a partial screenshot (e.g., “zoom in” effect with a circle or simply crop the original image).
If a more complex UI needs to be presented, simplify the original image so the user doesn’t try to read a 4px font size but instead sees simpler shapes.
Don’t scale down icons
E.g. for a favicon, redraw a simplified version instead: details will look fuzzy at a small size.
User-submitted content (e.g., Instagram feed)
Control the size and shape: crop images to keep the aspect ratio of containers so they don’t disturb layout (e.g., background-size: cover).
Add inner shadows or borders to user images to prevent background bleed.

Finishing touches

Use icons instead of bullets in a bulleted list.
Play with size and colors. E.g., you can have huge quotes around the block of text representing a testimonial.
Links deserve special styling.
Use custom checkboxes and radio buttons in forms: integrate the brand’s primary color(s).
Add color with accent borders, e.g. across the top of a card element, when highlighting active elements in the UI or to the left of alert messages.
Decorate backgrounds
- Change the background color. Works great for different page sections.
- Can use gradients: use two hues that are within 30˚ of each other.
- Use repeating patterns.
Consider the empty state carefully, especially when the user is meant to fill the UI with data.
- Prioritize empty states with illustrations and clear calls to action.
- I.e., a contact page with no contacts should feel welcoming, with some icon and button to add a contact instead of a blank page with an error-like statement.
- Similarly, things like dashboard may present a bunch of options that won’t apply until there is something to show: simplify the user experience.
Use less borders
- Use box shadows, spacing, and color instead of borders to distinguish elements.
- Have two background colors for containers (e.g., darker footer that looks separate from the main content).
- Replace borders with additional spacing.
Make components more exciting
- E.g., a table may not need all its columns if a particular column does not need to be sorted and could instead be part of another one (e.g., combining a dollar amount with a plan type or a person’s name with its title).
- Radio buttons could be cards instead if they are an important part of the UI.

Next level

Look for unintuitive decisions in designs you admire to find new ideas.
Rebuild interfaces you love to discover small details that make them exceptional.
Study others to build a solid foundation.

Conclusion

The concepts covered in this book demonstrate that effective visual design does not require innate artistic talent or formal training. By approaching interfaces methodically, establishing constraints through systems for recurring elements, and learning to strategically apply principles like visual hierarchy, white space, and depth cues, anyone can create designs that appear polished and purposeful. While internalizing these fundamentals does take practice, the ideas presented remove much of the subjective, nuanced expertise needed for things like picking complementary colors or balancing composition in illustrations.

By focusing on interfaces rather than free-form graphical work, the required skills become more structured and intentional choices drive outcomes. While studying formally can provide more breadth, the core principles in this text are enough to equip non-designers with the ability to build clean, intuitive, and aesthetically pleasing digital products. By implementing the recommendations covered, engineers and others new to design can gain confidence in their interface design skills. A great, concise read!

Building a task planner with React Native: an academic journey

Sébastien Lavoie — Sun, 30 Jul 2023 18:44:00 -0600

Preamble

For my last assignment in a BSc in Computer Science at the University of London, I decided to get involved in the world of task management to tackle one of the most common challenges usually offered to programmers, but with a twist based on scientific evidence. The result ended up being a cross-platform mobile application built with React Native, named ProductiviDo. Due to intellectual property concerns, I will not be able to share the source code of the application, but I hope you will nonetheless find joy in reading about the process of building it through the lens of the report I submitted for the assignment. For a complete list of references and appendices in context, please refer to the original report.

1 Introduction

The world needs a new kind of task manager, just like ProductiviDo, which comes to life through the “task manager mobile app” template provided for the Mobile Development module taught at Goldsmiths, University of London.

1.1 Inspiration: the main contenders

Inspiration.

There are a few wonderful products that deliver a good experience. Desirable features range from filters, labels and reminders to custom views and breadcrumbs. With Todoist, tasks can be quickly captured and will be added to an “Inbox” section. Once categorized, tasks are organized into projects, sections and sub-tasks.

Although Todoist recently added a board view, ClickUp takes it to the next level by labelling columns. In ClickUp, tasks are organized into multiple levels of hierarchies, down to task dependency, where sub-tasks depend on parent tasks.

ClickUp overview.

Trello takes an entirely different approach, where planning happens by moving cards onto different columns horizontally and re-ordering them vertically. Even though it has a concept of workspaces which is a group of boards, all the organization of a project usually takes place within a single board which contains multiple columns, each containing multiple cards.

Planning in Airtable is much more freestyle with a spreadsheet-like system. There is a Gantt view in the paid plan to visualize tasks over time. Using Airtable feels like interacting directly with a SQL database with its heavily grid-focused appearance.

Airtable project.

Finally, Asana shines mostly within a team setting and mainly works with the core concept of projects, which themselves can contain tasks, sub-tasks and even sub-projects.

Asana calendar.

1.2 Drawbacks of existing solutions

Drawbacks.

There are usability issues across all these mobile apps (and others) for different reasons. Most popular task managers have a limited search functionality, making it hard to find what one needs quickly unless a good organizational system has been consciously put in place beforehand. Crucially, such applications do not address the fact that unimportant tasks accumulate over time and provide no system to focus on what matters to avoid the fate of having “41% of to-do items […] never completed”.

A second point worth mentioning is that a calendar integration is not a core feature deeply integrated into these apps. For instance, Todoist offers a 2-way sync with Google Calendar, but tags, labels, priorities and areas of responsibilities are not shown nor taken into account in the calendar. Some task managers will provide their own built-in set of calendar features that partially solve the problem of being able to navigate a complete schedule in a breeze. However, none of these applications really allow efficient scheduling within the calendar. Some allow the user to add or modify events in the calendar, but this remains a slow process working with reduced functionality.

With regards to organization, all these applications are well-suited to manage tasks but when it comes to “time boxing” or “time blocking”, they will show every single task in the calendar view. This is impractical at scale as it quickly becomes overwhelming with dense information, a setup encouraging fragmented work and multitasking. Fragmented work caused by context switching has been shown to have a negative impact on productivity and on output quality while multitasking will lead to “experiencing more stress, higher frustration, time pressure and effort”.

1.3 Market overview

ProductiviDo will target in its first iteration Google Calendar because this product can be consumed from a plethora of desktop and mobile applications and there are about 4.3 billion Google users. Once data is integrated into Google Calendar, it can be consumed by different applications such as Apple Calendar, Zoom, Slack or other Google products, including Google Docs.

The number of smartphone users will increase by almost 23% over the next 5 years, forecasting more than a doubling over a 10-year period (2016–2026). Task management software is going upwards of USD$5 billion in market size, the CAGR —compound annual growth rate— is hovering around 14% and continuously growing. Combining supply and demand, we can see that both are expected to increase drastically over the coming years and that task managers fall into a category of growth market, which is desirable as it clearly indicates there is room for new rivals in the field: this is distinctly the case given that the task management world is still dominated by a few big players, many products have retired over the years (which unsaturates the market) and considering that ProductiviDo solves a real problem —that of staying on top of a growing list of commitments— with an innovative solution not yet implemented elsewhere.

1.4 Motivation

One reason why ProductiviDo will shine is because calendars are still under-appreciated in existing task managers, yet all major task manager applications integrate them in one way or another. It has been shown through many studies that calendars save time, improve productivity, reduce stress and make it simpler to batch similar activities. Plus, they can be consumed in a myriad of ways with their Application Programming Interfaces (APIs). The problem with using only a calendar without a task manager is that they are inconvenient when dealing with many events or tasks. On the other hand, relying only on task managers makes it difficult to appreciate how useful calendars are when it comes to visualizing one’s schedule from different perspectives such as a weekly or monthly view. Regarding ProductiviDo as a task manager and not only a calendar integration, it will distinguish itself by allowing users to quickly and efficiently record their routine activities. This is because it has been shown that these occupy a good amount of time even for productive people and there is currently no application that really implements a direct solution to this.

Intelligent prioritization is essential, yet difficult to achieve with existing products. Beyond capturing tasks and categorizing them, this application will feature a prioritization system that will help users focus on their most high-leverage activities with the help of custom sorting algorithms as advocated by Edmond Lau in his book The Effective Engineer. These will take into consideration aspects such as urgency and importance — in the spirit of the Eisenhower Matrix — as well as deadlines to calculate the impact that completing a task will have. Importantly, the priority matrix is adapted to account for other needed facets, such as when other people are involved in the task dependency chain (delegatees), an adaptation of the “fit” variable from the Sung Diagram, which is an extension of the Eisenhower Matrix.

The application will benefit users by providing a built-in planning system that will make it easier to follow a successful schedule. This goes hand in hand with findings illustrating how, in the absence of a mindful review process, users will want to focus on small or unimportant tasks to release dopamine even though they know what would bring them the best outcomes, a real productivity threat leading to a decrease in output and an increase in completion time. Therefore, by offering a system that will encourage users to take better decisions and visualize evidently their highest-leverage tasks in a familiar and useful fashion, ProductiviDo will carve its place in the market by solving unmet needs.

2 Literature review

2.1 Previous work

As alluded to in section 1.1, the current work found inspiration in well-established task management software. There are numerous alternatives available on the market: only the most well-known and relevant ones for this project will be covered. Evaluating such products in depth goes beyond the scope of this report: key points will be highlighted to justify the need for a new solution.

2.1.1 Todoist

Todoist is one of the most popular task management software, with over 20 million users. It is a web-based application that allows users to create tasks, organize them into projects, and assign them to specific dates. It also features a priority system that allows users to sort tasks by importance, and a reminder system that will notify users of upcoming tasks. While Todoist is a very popular solution, it has some shortcomings. Its design is arguably polished, yet its dark theme lacks contrast, recurring tasks don’t show immediately on which schedule they recur and tasks with comments or attachments are difficult to locate. Other functional deficiencies include the fact that the “Upcoming” view shows all tasks from all projects as a large, single list of items without allowing filtering or that the app has a concept of “due date” with no “start date” such that duration is not taken into account nor can be visually represented meaningfully in a calendar. Equally relevant when comparing to what ProductiviDo will offer is the fact that there is no way to apply “time boxing” as there is no calendar view and only a functionally limited Google Calendar integration, task filtering is not intuitive due to the custom query language syntax used by the app and there is no easy way to find recurring or routine tasks without setting up a custom labelling system on one’s own.

Todoist upcoming view.

2.1.2 Trello

Trello lies on the other end of the feature creep spectrum when compared to ClickUp and can be seen as possessing an even more basic set of capabilities than Todoist. With its simple and uncluttered drag & drop interface, Trello is much more intuitive than a more sophisticated and flexible option such as Todo.txt which can be technically extended at will and can be operated from the command-line on a desktop computer or from a multitude of third-party applications on mobile devices. Trello’s simplicity comes at a cost, however: it is not uncommon to lose track of “cards” because one must swipe horizontally in order to navigate the UI; and very little information fits on the screen at once, where only a fraction of a “board” with up to a few “cards” is shown. There is a copious amount of calendar integrations available for Trello, yet the core of the issue that ProductiviDo aims to address remains unsolved: the app’s metadata is not accessible from Google Calendar (e.g., priority, area of work, task status, etc.) and the app does not provide a way to visualize the user’s tasks in a convenient calendar view as it only “displays all cards with due dates by month” and only when using one of its paid feature, called “Power Ups”.

Trello boards.

2.2 Contributions from the literature and gaps being filled

2.2.1 Initial design justifications

Amongst a panoply of scientific papers on the subject of task management, the quintessential work of Bellotti et al. serves as an excellent entrypoint listing a set of design requirements to be expected in an efficiently conceived task manager. Of particular relevance to the creation of ProductiviDo are the needs for the app to be “instantly on” with the option to have “no formal task description” and a “mechanism for handling stale to-dos of low importance”. The authors highlight that the main issue is “making sure that the important tasks get done”, which led to further research being exposed on the topic of prioritization matrices. The authors also advocate for task states, location, social relations, notes and time constraints to be captured, all of which are taken into account in the design. In a more recent study looking at the importance of prioritizing tasks, further evidence suggested that a prioritization model should be adopted (e.g., HI/LO, CARVER or Carpenter) in a task management system, pointing at the Eisenhower Matrix as a useful framework to use for this very purpose. Other less formal alternatives such as the RICE or ICE frameworks have also been reported to be used with success and could be adapted to this project with their scoring model.

Another aspect which served to inform the overall direction to take is the well-known fact that humans do not work optimally when multi-tasking as this leads to a high cognitive load in keeping track of different areas of focus or projects. Fragmentation of information in one’s work —which occurs when multitasking— is a problem because people won’t remember where to find the information. Applying this idea of fragmentation to the digital world, it only makes sense that offering too many ways to store and retrieve data (such as with ClickUp) is counterproductive. Furthermore, the ability to customize the UI is a double-edged sword because “users make mistakes and create building blocks with unintended consequences”. Indeed, if the user interface is cluttered or offers too many bells and whistles, managing the global context of one’s tasks becomes difficult and one inevitably encounters obstacles when trying to juggle with multiple actions at once, because “prospective memory is fallible”. Hence, simplicity is key to a good task manager (which certainly helps to explain Todoist’s wild success). While simplification is part of the gamified model —Habitica being one example of a product incorporating many elements of gamification— this model suffers from the “undifferentiated use of rewards”, leads to “punishments for productivity”, to “feeling of not being taken seriously” and cause “negative anticipation”. Moreover, “people found their standard task manager simpler to use” and the integration of extrinsic motivational factors such as a reward system “would not excite them to continue being immersed in a gamification application”. While this form of motivation does not appear to be effective in goal attainment, the literature concludes that “valuing extrinsic goals […] does not seem to increase our happiness, but attaining those goals does”, adding that giving weight to intrinsic goals (such as personal growth, loving relationships and physical health) and achieving them is a better way to increase happiness. As long as a task manager nourishes intrinsic motivation, it is likely to be effective in helping people to achieve their goals while maintaining optimal psychological health.

Other approaches were considered, notably the “Binary Priority List” due to its simplicity when it comes to comparing two elements (such as tasks). However, it lacks flexibility and becomes inefficient as more and more tasks are added because —unlike the famous binary search algorithm— this algorithm requires human intervention to classify tasks, which becomes burdensome faster than O(log n). The literature on gamification also linked to sources that revealed solutions to solving procrastination, a self-regulation failure that gamified systems sought to minimize to no avail, which led to identifying a desirable core component of the application: implementation intentions. These can be integrated into the “mental flexibility” facet of “planfulness” —through a technique known as “mental contrasting with implementation intentions” (MCII)— along with temporal orientation and cognitive strategies, all of which improve goal outcomes.

2.2.2 Implementation intentions

Implementation intentions have shown their effectiveness in helping people to achieve their goals, the real purpose behind using task managers in the first place. The idea is to create a plan of action for a specific goal, which is then followed by a specific time and place. The plan must be broken down into a series of steps, which are written down in a way that is easy to remember using so-called “if–then plans”. The specific formulation of such plans “produce much more favorable and reliable outcomes than broad intentions to pursue a goal”. Findings from Bieleke et al. show that implementation intentions are also useful when planning one’s work because they help “evaluate information from a certain perspective” —which can be verbally recommended to users so they adopt a “certain processing style (e.g., deliberative thought) or perspective (e.g., neutral observer)”. Their research also highlight the high potential of implementation intentions in reviewing tasks because “participants can strategically adopt a reflective mode of information processing that helps them make more sophisticated decisions when facing uncertainty”. Other studies show that such “plans to deliberate can be used to increase the likelihood of deliberation and thereby the effective processing of newly available information”, which further supports the benefits of implementation intentions with regards to planning. To make the planning stage more effective, research also “indicates that having plans that focus on overcoming obstacles are important for maintaining progress and staying on track with one’s goals”, so this aspect will also be featured when creating tasks in the “Inscribe” step described in section 2.3.1. Once planning has been done, implementation intentions will also prove useful to take action towards accomplishing one’s goals because they lead to “automatic action initiation without further conscious intent” —surprisingly and encouragingly, even in participants suffering from mild to moderate depression. While this is obviously beneficial during the initial stages of a project or task to overcome procrastination, it is equally important to re-evaluate one’s course of action because sticking to a plan stubbornly “can hinder overall goal performance by causing one to overlook alternative opportunities for achievement”, amongst other negative consequences. This explains why ProductiviDo encompasses a “Review” step which deals with this particular problem and more, described in section 2.3.4.

While this self-regulation technique can be applied successfully in the context of a task manager, it bears mentioning that it is not a panacea as it suffers from some limitations, one of which being that “formulating multiple implementation intentions is ineffective when changing unwanted behavior […] due to interference in the enacting phase of the planning process”. In other words, multiple implementation intentions can compete for attention, which makes one lose focus. To drastically mitigate this issue, ProductiviDo dedicates an entire step to “Focus”, which is described in more detail in section 2.3.3. The “Review” step will help with reducing the number of tasks one has to juggle with while there will be a “Prioritize” step (described in section 2.3.2) to help users decide which tasks to focus on before they get conveniently extracted in the “Focus” step.

2.2.3 Calendars, planning & scheduling

Section 1.4 introduced some of the core benefits of using calendars for planning and scheduling, yet scientific examination supplies a treasure trove of additional findings justifying their use alongside a task manager. A study exploring the use of a personal calendar discovered that most interviewees prefer a weekly view to aid with “opportunistic rehearsal”, a fact that sits well with the need for a reviewing system such as the one proposed in section 2.3.4 as well as the realization that a simple list or Kanban presentation is helpful but not sufficient to get a clear overview of one’s schedule at a glance. The same study made it clear that most users rely on their calendar to store efficiently certain types of information such as the date (97%), time (96%), location (93%) and purpose (69%). Quite convincingly, participants also used their calendar for “tentative event scheduling”, revealing the need to empty their mind to stay on top of their duties, a need addressed with the “Inscribe” step referred to earlier. This study also emphasized the fact that calendars can be viewed on the web from most devices or even printed for offline reference. Finally, this paper also found that a majority of users (63%) reported using reminders and alarms with the help of their calendar.

Even though almost two thirds of working professional consider using a calendar application to be very important in their workflow, it was also found in another study that “it is inconvenient to schedule on calendar apps”, which supports the use of task managers for that specific necessity since they can alter tasks brilliantly and speedily. Calendars are not necessarily the best tactic when it comes to scheduling all kinds of activities either. Indeed, “when consumers schedule their leisure, they may inadvertently reduce their utility for the activity”. With this type of activity, the authors divulge that “roughly scheduling (i.e., without pre-specified times)” constitutes a practical alternative. This results in effective scheduling when tasks are entered via a task manager and synced with Google Calendar since blocks of activities can still be displayed in the calendar view without the added burden of specifying a strict duration for each event.

There are some use cases that simply are not well suited for task managers, such as synchronizing events from Gmail (i.e., capturing tasks from different sources), setting reminders on appointments scheduled by third parties (where the primary source of truth must remain the calendar itself) or sharing a schedule with groups across the G suite applications with the help of smart suggestions coming from Google users’ data which is not entirely available to external applications.

2.3 The four pillars forming a new approach: evaluating their effectiveness

2.3.1 Inscribe

Inscribe screen.

ProductiviDo features four core components to help tackle one’s work skillfully, the first of which being all about jotting down tasks, a demonstration of the “generation effect” which assists with remembering information better. At this stage of the process, the idea is to record any potential future action that is required while being fully aware of the impact that such action will have on one’s life, benefitting at once from reducing one’s cognitive load by storing or discarding thoughts that do not require immediate response. “Inscribing” in this application is an intentional step demanding a certain level of mindfulness and acts as a precursor to the use of implementation intentions as it similarly “serves an important self-regulatory function” according to a study on mindfulness done at the University of Rochester. A 2020 study from Bieleke and Keller found that “opportunity-focused plans are especially important for initial progress towards the goal”, therefore this step ensures —through questioning the user’s aspirations— that the task is indeed a priority and that it is not just a distraction by evaluating its “leverage”, a notion introduced in section 1.4.

2.3.2 Prioritize

Prioritize screen.

There is substantial evidence to establish that the concept of a “master list” has been popularized and has found multiple concrete applications (e.g., as a list of relevant jobs held by an applicant, as an Excel spreadsheet keeping track of crucial information at a high-level, in some jurisdictions in the legal world, etc.), including as a way to manage tasks, an idea also sprouting from Edmond Lau’s work. In the context of task management systems, a master list manifests itself as a centralized location where all tasks are accessible once they have been “inscribed”. With ProductiviDo, “prioritizing” is a transient state: it is about handling a database of data to be referenced later when progress must be undertaken on specific tasks. It is a digital space separate from the current areas of focus (presented in the following step in section 2.3.3) used to find the next priorities to work on, where filters are available to that end (see section 3 for the technical details). As defended by Bellotti’s research as a set of desirable features, the “Prioritize” step offers the “ability to sort by importance and deadlines” as well as a “smart prioritization” of tasks by default, where the most important ones will flow to the top of the list, ready to be acted upon. Because tasks listed in the “Prioritize” step are forcibly distinct from those listed in the “Focus” step as they are mutually exclusive, this intermediate step helps to reduce the number of tasks shown in the schedule, improving one’s perceived control of time, busyness and stress.

2.3.3 Focus

Focus screen.

Once there are prioritized tasks in the application defined in terms of implementation intentions, science dictates that there is a need to “support the viewing of entire task vistas, but also allow different perspectives for different kinds of planning” and for “top priority items to be made apparent”. It is also known that setting clear goals eliminates intrusive thoughts and that “commitment to a later behavior was linked to reduced rather increased strain on mental resources”. For these reasons, the “Focus” step is a crucial component of the application. This component has the additional benefit of mitigating the problem uncovered in section 2.2.2 about the lesser efficacy of having multiple implementation intentions at once because it helps to recall all the high-leverage tasks in a filtered version of the master list (i.e., the current working list), leading to the observation of the Zeigarnik Effect which states that “an activity that has been interrupted may be more readily recalled”. In addition, seeing a list of promises to oneself in the form of MCIIs “seems to facilitate behavior change even when there is an initial reluctance to engage in the targeted behavior”. Even more benefits from MCIIs include “sustained task value and higher persistence”, two aspects that contribute to goal achievement and, consequently, to turning a task manager into an efficacious product.

2.3.4 Review

Review screen.

At this point, a working list of tasks exists to “Focus” and a backlog of work remains to “Prioritize”, yet not all labor should be brought to fruition as initially planned. Life circumstances change and “plans to deliberate can be used to increase the likelihood of deliberation and thereby the effective processing of newly available information”, which can help eliminate or update tasks that are no longer considered important or relevant enough given an ever-changing environmental context. A paper on making informed scheduling decisions also noted that “self-monitoring and self-reflection often affect behavior, and this change typically goes in the desired direction of improvement”. These findings can be applied practically by creating plans focusing on overcoming obstacles to make sustained progress, allowing one to ponder every major decision during the journey. A 2021 study on task management tools proposed to break complex tasks into smaller chunks (using a sophisticated approach with dependency graphs), an idea that can auspiciously be employed in a simpler way by breaking down a larger unit of work into smaller ones, which can be done during the information-gathering phase (i.e. the “Inscribe” step) as well as during the reviewing phase with existing units of work, which is known as “compartmentalization”. This self-reflection process can also contribute to one’s well-being by providing written cues to recollect positive images from past accomplishments. Looking towards the future, cueing the users to imagine positive outcomes and encouraging them to restructure their priorities can equally evoke optimism and improve life satisfaction while inspiring them to live according to their personal values. After all, a “person who sets up life and its routines to avoid inner conflict between goals is better off in the sense that he or she ends up feeling fewer bad emotional states and is generally happier”.

Review screen, showing a filter being applied.

3 Design

3.1 Domain and target audience

ProductiviDo is a task management system staying clear of project management and time management, geared towards being a capture and review tool leaving notifications and scheduling to external systems, primarily to Google Calendar in these first product iterations. As such, the target audience is represented by conscientious people desiring to optimize their productivity with a tool discouraging multitasking, who don’t mind hand holding to achieve their goals — willing to trade off flexibility in exchange for a more rigid system backed by science —, expecting a simple set of essential features for a distraction-free environment to get things done and wanting to manage their own set of personal tasks (as this product is not designed with teams or businesses in mind).

Explicitly supported users include people who want to follow a useful system without having to build it from scratch (e.g., setting up projects, tags, custom priorities, etc.) as well as those who need to keep track of tasks involving others with basic functionality (via Google Calendar and filters in the application). The application will not cater to those who want to rely mainly on desktop applications (because the core of the application is mobile only), nor to those who want to use a calendar other than Google Calendar (which is the only calendar undeniably supported in this version) or those who are looking for a solution integrating into other applications, since there will be no publicly accessible API at first to make this possible.

3.2 Design decisions and principles

Designing and wireframing in Figma.

Section 2 abstractly laid out the foundation for the core ideas being implemented in the application meeting the needs of the target audience. Concretely, a few design principles listed in “The Pocket Universal Principles of Design” inform how the application has been thought out, markedly:

The 80/20 rule: “80 percent of a product’s usage involves 20 percent of its features”, which is true with ProductiviDo where users will spend most of their time focusing on the work they have to accomplish;
Chunking: “Chunk information when people are required to recall and retain information”, where each task will display the most important information to contextualize work;
Constraint: “Limiting the actions that can be performed to simplify use and prevent error”, where each screen will have a single function with a premedidated lack of customization;
Five Hat Racks: the five ways in which information can be organized (i.e., category, time, location, alphabet, and continuum), all of which are supported by ProductiviDo;
Form follows function: “Aesthetic considerations should be secondary to functional considerations”, where the application maximizes the functionality available on each screen while taking into consideration the primary interactions needed at each step, regardless of the visual impact these decisions have;
Garbage in, garbage out: “Make it impossible to store bad data types”, where the application will not accept invalid inputs on any field that could jeopardize the integrity of the data;
Nudges: “smart defaults, clear feedback, aligned incentives, structured choices, and visible goals”, where the app makes the purpose of each interaction crystal clear;
Progressive disclosure: “A method of managing complexity, in which only necessary or requested information is displayed”, where filters are selectively shown to match the intent of each screen and where optional fields are hidden away by default until they are needed.

It is noteworthy that calendar integration will be optional and that the application will remain fully functional without relying on external calendars for maximum convenience, especially while users are offline. Even though this work underlined the usefulness of calendars when used as companions for task managers, one of the primary goals of a task manager is to be available at all times regardless of internet connectivity so that users can access their list of priorities and get work done: requiring an active connection and access to a calendar via sign-on before being able to do any work would break the principle of least astonishment in this type of application.

Some of the original wireframes for the Prioritize screen.

3.3 Structure of the application

The application is structured in such a way that each screen has a single function and that the user is guided through the process of adding, editing, and reviewing tasks4. The application is designed to be as simple as possible, with a minimalistic interface and a limited number of interactions to keep the user focused on the task at hand. Technically, it is arranged according to a pyramid schematic architecture, where the “hub” is the hamburger menu which links to all other pages and where all the sub-pages can go back to the “hub”. In this architecture, the tab navigation at the bottom of the screen lists the core sub-pages which are always accessible and can lead back to the hub as needed so that the user does not need to remember how to get from one screen to another, since they are all identified with both a recognizable icon and text label.

The application is structured in a way that makes it easy to add, edit, and review tasks, with the following screens:

Inscribe: Looking at the tab bar (bottom part of the wireframes) from left to right, this is the first screen meant to be easily accessible and also the first step that is part of the process of managing tasks by carefully adding them to the system;
Prioritize: this is the screen responsible for displaying the list of tasks that have been added to ProductiviDo in the first step (Inscribe) but still require planning (i.e., they represent the tasks to prioritize);
Focus: Once the tasks have been prioritized from the previous screen (Prioritize), they are ready to be scheduled and worked on, which is the purpose of this screen — it displays the current working list of tasks in a digestible format;
Review: Periodically (daily, weekly, etc.), the application will ask the user to review the tasks that remain to be completed to ensure that they are still relevant and that they are still a priority.

Edit task screen.

These core screens are accessible at all times from the tab bar to allow for easy navigation between them. The application will also have a few other screens that are not part of the core workflow but that are still important for the user experience:

About: This screen will display information about the application, such as the purpose behind each aspect as well as the version number;
Help: There will be a help system embedded in this screen to help users understand how to use the application and how to get the most out of it, which will be fully searchable;
Recycle bin: Sometimes, tasks will be added to ProductiviDo by mistake or they will be completed before they are scheduled. This screen will allow the user to review and restore these tasks if they are still relevant;
Accomplishments: This screen will display a list of tasks that have been completed and will allow the user to review them and reflect on their accomplishments;
Settings: This screen will allow the user to customize the application to their liking, such as changing the theme, determining default values on new tasks, etc.

About screen.

These secondary screens are accessible from each core screen in the top-right corner to toggle the presence of a “drawer” menu. The menu can also be revealed by swiping from right to left on the screen. The menu will be hidden by default to keep the interface as clean as possible and to avoid cluttering it with unnecessary information. The menu will be revealed by default on the first launch of the application to help users discover the features available to them.

Menu screen.

3.4 Libraries and services

At a high level, the application will be built using React Native to support both Android and iOS platforms. In terms of software user-facing dependencies, the application will use the following libraries:

react-native-async-storage/async-storage: The application will use this library to store data locally on the device;
react-native-community/datetimepicker: This library will be used to display the date and time pickers;
React Navigation: This library will be used to implement the navigation system;
Redux: This library will be used to manage the state of the application;
NativeBase: it will be used to implement the user interface;
redux-persist: This library will be used to persist the state of the application locally on the device;
redux-thunk: This library will be used to implement asynchronous actions in the application;
react-native-collapsible: This library will be used to implement all the collapsible sections to provide progressive disclosure;
react-native-extended-stylesheet: This library will be used to implement the theming system;
nandorojo/anchor: This library will be used to implement the anchor system so that users can jump from one section to another in the help system.

The application will also use the following libraries to support the development process:

Eslint: This library will be used to enforce a consistent coding style;
Babel: This library will be used to transpile the code to support older versions of JavaScript;
Jest: This library will be used to implement unit tests;
react-native-testing-library: This library will be used to implement integration tests;
react-native-dotenv: This library will be used to manage environment variables;
react-test-renderer: This library will be used to implement snapshot tests;
TypeScript: It will ensure type checking is available app-wide and reduce the number of bugs.

On the backend side, the application will make use mainly of the following services:

Firebase: This service will be used to implement the authentication system needed to deal with Google accounts as an optional add-on;
Google Calendar: This service will be used to implement the calendar system;
SQLite: It will be used to implement the database system locally and will be backed up to the cloud.

Showing the database design behind ProductiviDo.

3.5 Plan of work

In a nutshell, the plan of work will be followed as shown below:

3.6 Testing and evaluation

The project will be tested primarily using the following methods:

Unit tests: Unit tests will be implemented to ensure that each component of the application works as expected. These tests will be implemented using Jest and react-test-renderer.
Integration tests: Integration tests will be implemented to ensure that the application works as expected when all the components are combined. These tests will be implemented using the apt react-native-testing-library.
User and usability testing: User testing will be implemented to ensure that the application is easy to use and that it meets the needs of the users. These tests will be implemented using a combination of user interviews and surveys.
Performance, load and stress testing: Performance testing will be implemented to ensure that the application is fast and responsive. These tests will be performed by simulating the presence of a large number of tasks and events.
Security testing: Security testing will be implemented to ensure that the application is secure, most definitely when it comes to protecting user’s data in the cloud to avoid data breaches.
Regression testing: Regression testing will be implemented to ensure that the application does not break when new features are added, using a mixture of automated git bisect scripts to locate anomalies and by automatically running test suites with Git pre-commit hooks.
Compatibility testing: Compatibility testing will be implemented to ensure that the application works on all the supported platforms.
Accessibility testing: Accessibility testing will be implemented to ensure that the application is accessible to all users.

4 Implementation

4.1 Overview

At its core, ProductiviDo aims to provide a system that facilitates “getting things done” in the manner of the famously successful productivity system GTD, created and shared with millions of people by David Allen. The system is based on the following principles, worth expanding on as it bears strong resemblance with the final design of the application:

Capture: All tasks and events should be captured in a single place, so that they can be easily accessed and managed. This is achieved with the “Inscribe” screen, the first step in the ProductiviDo system.
Clarify: All tasks and events should be clarified, so that they can be easily understood and prioritized. This is achieved in the “Inscribe” screen, where the user can add a description to the task or event along with necessary metadata to help classify and prioritize the data being added.
Organize: All tasks and events should be organized, so that they can be easily accessed and managed. While this GTD step does not map directly to a step in ProductiviDo, the “Focus” screen serves a similar purpose in that it allows the user to organize tasks and events into a list of immediately actionable items that are easy to reference.
Reflect: All tasks and events should be reflected upon, so that they can be easily understood and prioritized. With ProductiviDo, this is done in the “Review” screen, which invites users to review tasks logically according to their deadline, priority, and other metadata.
Engage: All tasks and events should be engaged with, so that they can be easily accessed and managed. ProductiviDo deals with this GTD step in a simpler way via the “Prioritize” screen and the “Focus” screen, which allow users to prioritize tasks and events and then focus on the most important ones.

4.2 Features implemented: a visual tour

Quick actions.

The application has the following features:

Add a task: The user can add tasks by navigating to the “Inscribe” screen and filling in the required fields. The user can also add a deadline date and time, a duration, a location, a priority to the task and so on. By choosing the type of task as “Calendar” and filling in the required fields, the task will be added to the user’s Google Calendar. This screen hides any navbar and tab bar to maximize the screen real estate.
Prioritize a task: The user can prioritize tasks by navigating to the “Prioritize”, where filtering options are available to help the user find the task to prioritize. The main task action that can be performed on this screen is to “focus” a task, effectively moving it to the “Focus” screen.
Focus on a task: The user can focus on a task by navigating to the “Focus” screen, where the user can see a list of tasks that are immediately actionable. The user can also filter the tasks to focus on. The main action that can be performed on this screen is to “complete” a task, effectively moving it to the “Accomplishments” screen.
Review tasks: The user can review tasks by navigating to the “Review” screen, where the user can see a list of tasks that are overdue, due soon (e.g., today, tomorrow, this week), due later (e.g., next week, next month, next year) as well as tasks that are “old” (i.e., tasks that have been added to the system more than a month ago relative to today’s date but have not been worked on yet). This screen features “quick actions” that are relevant to the tasks being reviewed. These actions provide the ability to reschedule a task, update its current state (i.e., move it from and to the Prioritize and Focus screens) and to delete a task without having to open the task details screen to edit it.
Filter tasks to review: The user can filter tasks to review by navigating to the “Review” screen and selecting the filter options. The user can filter tasks by one of the categories “Overdue”, “Soon”, “Later”, “Old” and by all of these at once when no filters are applied.
Edit task details: Once a task has been added to the system, it can be edited from any of the main screens (i.e., “Prioritize”, “Focus”, “Review”). The user can edit all the same properties as when adding a task as well as add new ones not present when the task was added. When a task is of type “Calendar”, its related details will also be updated on the user’s Google Calendar.
About screen: while the About screen does not provide any functionality related to tasks, it allows users to learn more about the application. The screen includes a link to the application’s website, a link to a dynamic feedback form to help ProductiviDo become a better fit for its users, an email address to contact the developers (which opens the default email client on the user’s device) and other less frequently accessed information, including a link to the GitHub repository of the project and a list of open source libraries used to create the product.
Help screen: the application has been designed to avoid relying on documentation in order to be used. However, the Help screen is a useful resource for users who want to learn more about the concepts involved in the application. It describes possible workflows and guides users through the application’s main screens so they will be able to use it even more effectively.
Recycle bin screen: ProductiviDo makes it very easy to access and manage tasks that have been deleted by the user. The Recycle bin screen allows users to restore deleted tasks (which are sent back to the Prioritize screen for the next round of prioritization) or to permanently delete them by emptying the recycle bin.
Accomplishments screen: While the Recycle bin holds all deleted tasks, the Accomplishments screen holds all tasks that have been completed by the user from the Focus screen. The Accomplishments screen is a great way to keep track of the user’s progress and to celebrate the user’s accomplishments, yet it also allows removing completed tasks from the system if the user wishes to do so.
Settings screen: The Settings provided by ProductiviDo allow users to customize the application to their needs in a basic way — and all settings are persisted across reboots of the application or device. There is a dark mode available to protect the users’ eyes from screen brightness. Other settings include the ability to set default behaviours on new tasks, set the increment in minutes to be shown in the time picker, set a different locale to represent dates and times and some more options to toggle the display of certain elements in the application.
Filters: What ProductiviDo lacks in customizations, it makes up for in filters. The application allows users to filter tasks by type, status, priority, deadline date, deadline time, duration, location and more. The filters are applied in real-time, so the user can see the results of the filters as they are being applied. The filters can be applied to any of the main screens (i.e., “Prioritize”, “Focus”, “Review”) and can be combined to create more precise filters.
Google Calendar: One of the key tenets of ProductiviDo is to make it easy for users to manage their tasks and to have a visual window into their schedule, which is achieved by integrating with the user’s primary calendar on Google Calendar. The application allows users to connect their Google Calendar account to ProductiviDo and to create calendar events from new or existing tasks. At the same time, this functionality remains entirely optional: users can choose to never opt in to the Google Calendar integration and can disconnect their Google account at any time, which will prevent any synchronization from happening between ProductiviDo and Google Calendar.

Settings screen.

Dark mode.

4.3 Techniques and methods

Help screen.

Beyond the libraries mentioned in section 3.4, the actual implementation of the application happened in the following way:

React Native bare workflow: The application was developed using the React Native bare workflow, which allows for the use of native code in the application. This was necessary to implement the Google Calendar integration, which was not possible using the Expo workflow.
State management: The application was architected in such a way that React Context was used to avoid “props drilling” and to provide a global state to the application, using data coming from the SQLite database loaded at launch. All context that does not require persistence is managed by React Context, while the rest of the state (e.g., settings) is managed by Redux Toolkit.
TypeScript: The app was developed using TypeScript, a typed superset of JavaScript that compiles to plain JavaScript. TypeScript was used to ensure type safety and to provide a better developer experience with autocomplete and other features that help developers write code with fewer errors.
React Navigation: The navigation between screens uses a combination of stack and tab navigation, with the main screens being accessible through the tab navigation and the rest of the screens being accessible through the stack navigation. Additionally, a drawer navigation is used to provide access to all secondary screens via the traditional “hamburger” menu.
NativeBase: This library was used to provide a consistent look and feel to the application and get access to components that are not available in React Native. Many of its components were put to use in the application, including Actionsheet, AlertDialog, Badge, FlatList, Modal, Switch and Transitions to name a few.
Google Calendar API: The application uses the Google Calendar API to allow users to connect their Google Calendar account to ProductiviDo and to create calendar events from new or existing tasks. The API is used to create new events, to update existing events and to delete events. The API is also used to retrieve the user’s primary calendar and to retrieve the list of calendars available to the user.
SQLite: The application uses SQLite to store all data locally on the device. The application uses the expo-sqlite library to interact with the SQLite database. The database is initialized at launch and is updated whenever the user makes changes to the data.
Form validation: The application uses the yup library to validate forms and to provide error messages to the user. Coupled with formik, the library provides a great way to validate forms and to provide a consistent experience to the user.
Custom hooks and functional components: This project explicitly embraces new React features such as custom hooks and functional components. The application uses custom hooks to encapsulate logic for all the main screens. The application also uses functional components to avoid the use of class components, which are considered legacy in React.

5 Evaluation: results and discussion

5.1 Successes

5.1.1 Planning and execution

Extensive research and preparation went into the project before writing a single line of production code. A multitude of potential implementations and concepts were explored in great details and the idea of the final product was refined from a very early stage, leading to a more focused development of the todo application. Because there existed a tangible vision of the desired outcome, it made it much simpler to define the database schema and the React context needed throughout the application to pass adequate data structures around. On a related note, doing all this work upfront helped to avoid wasting time learning libraries that would not entirely meet the needs of the application and to stop contemplating features that would not fit within the constraints of the chosen concept.

Another decision that proved to be quite useful was to postpone dealing with complicated services till much later during development. For instance, storing mock data as simple JSON allowed for quick iterations during development without having to manually add tasks one by one by using the application itself to do so. Once it came time to integrate database functionality, data was first stored as a temporary in-memory database for fast read/write speeds (because the task view, filtering and sorting was not optimized yet) and it was later persisted to disk. Likewise, external services (e.g., Firebase and Google Calendar) were only integrated until the internal application design was set in stone. Performance optimizations were favorably delayed until most of the logic was implemented, which gave ample time to test and refine the design, productively steering clear of premature optimization due to the changing nature of the application still being validated and tested by beta users.

5.1.2 Functionality achieved

One concept that served the final design well was a direct application of Fitt’s law, which states that “the time required to touch a target is a function of the target size and the distance to the target”, meaning that it is important to keep controls close and large when speed or accuracy matters. As a concrete demonstration of this law, tasks can be added with the touch of a single button in the “Inscribe” button located in the tab bar, making it frictionless for users to add tasks and events to the application. This meets the need of being “always on”, especially since all data is accessible locally without the need for an internet connection, allowing users to quickly get their priorities out of their mind and onto a digital device to allow for proper referencing later on. Other critical interactions such as focusing tasks and completing them can be done by tapping on a large icon next to each task.

The application allows visualizing all tasks at once in the “Prioritize” screen, which contains all necessary filters in order to make filtering and sorting through a growing list of incoming data not only feasible but also simple and practical. Indeed, the lack of hierarchical organization in the information architecture is absolutely intentional. This design decision allows flexibility with the use of filters without incurring a cognitive cost to understand how information is organized. As long as users are diligent with the metadata they input in the application (as they should if they wish to accomplish their goals), they will be able to easily find the information they need in the “Prioritize” screen. Excluding the hierarchical access to information, all other aspects of information architecture (location, alphabet, time, category) are available as filters to the users.

The “Focus” screen intuitively lets users know which tasks are most important to work on at any given time, sorting them by deadline, priority and other metadata, making the data “always actionable”. Since focused tasks are isolated from the rest of the data, users are able to focus on the task at hand without being distracted by other tasks. On this screen, users are able to complete tasks, which effectively disappear from the task list and are kept in the “Accomplishments” screen for later retrieval. This is a way to keep track of tasks that have been completed, which is a feature that is not available in the Google Calendar application and which complements its functionality very well. Additionally, this screen allows users to “restore” tasks, which performs the opposite action. Notably, it will prevent mistakes by letting users undo their actions; it will also allow users to review tasks and events that have been deleted and to permanently delete them as needed. This feature is not available in the Google Calendar application and contributes to making ProductiviDo a useful ally in this case so that events can be restored to the calendar view. The final active step required to use ProductiviDo effectively, found in the “Review” screen, provides complimentary functionality to that found in most modern calendar applications. While a calendar is useful to get a sense of upcoming events, a review system is needed to stay on top of one’s priorities to make sure that time-insensitive yet important events do not slip through the cracks. The “Review” screen is a way to make sure that users are not only aware of the tasks and events they have scheduled, but also of the tasks and events that they have not scheduled but are important to them. Furthermore, reviewing one’s work periodically in an automatic way allows for better planning and will make sure that longstanding tasks are either updated, prioritized, delegated or removed.

5.2 Failures and limitations

The following points of failures are discussed in more details in section 6.2, where solutions for brighter times ahead are pictured.

5.2.1 Google Calendar integration

ProductiviDo solves real-world issues related to task management and scheduling in a robust manner, yet it does so partially. One major caveat to the current implementation is that it supports only Google Calendar, which severely limits the potential market the application can tap into. Because only Google Calendar was taken into account during this project, it is highly likely that the application will not be able to support other calendar applications without a major overhaul of the codebase. The application was thought to be a companion to Google Calendar from the ground up, which means that while it integrates seamlessly with it, it creates tight coupling between the two applications. There are in fact more issues to delve into when it comes to Google Calendar. In particular, full synchronization of tasks and events is not supported in a scalable way. For one, Google Calendar API’s daily and monthly rates are limited: instead of calling that API on every little detail being updated in a task, the application should really try its best to batch requests so that heavy users will not be penalized if they happen to trigger many updates in their calendar or if Google decides to lower the number of requests that can be processed in a given amount of time. Another pitfall is that the application does not support recurring tasks as originally intended due to lack of human resources, which is a feature that is available in the Google Calendar application. While it is technically possible to implement this feature, more context would need to be stored in the application to avoid duplication of tasks in the database and to allow recurring tasks to be reset to their original state or updated as desired, all of which would need to communicate with the Google Calendar API in an efficient way. There is, at the moment, no task tracking facility: a completed task is simply removed from the list of tasks by updating its status in the database. This is a considerable limitation, since it prevents users from creating tasks that are recurring, such as weekly meetings or monthly bills.

5.2.2 Unit testing

One clear oversight in the development process was the lack of unit testing. While the application was profusely tested in a manual way, it would have been much more efficient to have had a suite of unit tests that would have allowed for a more sturdy and scalable application. ProductiviDo comprises a few essential unit tests regarding the Google Calendar integration, but it is nowhere near the level of testing that would be required to ensure that the application is robust and that it will not break in the future. Amongst some of the types of tests that should have been implemented are unit tests for the database, unit tests for the application’s business logic, snapshot testing and testing of asynchronous API calls through mocking. In hindsight, the reason more tests did not make it into the final application is that the development process was not agile enough to allow for the time required to write unit tests and there have been too many side experiments that have taken priority over the core functionality of the application. Of equal relevance is the fact that unanticipated issues have arisen during the development process, which have required more time and effort to be spent on fixing them rather than on writing unit tests. It is also important to keep in mind that unit testing is not a one-time activity: it should be done continuously throughout the development process, more akin to the test-driven development (TDD) workflow. Failing in that department was a major setback for the application, since it will make it harder to maintain and to scale in the future and it likely contributed to the fact that more time was spent on debugging than was necessary.

5.2.3 Advanced features powered by AI

Besides the issues already raised in this section, there are a few other features that would have optimized the user experience, but which were not implemented due to lack of time as otherwise the scope of the project would have fallen appreciably out of reasonable proportions. For instance, the application does not support batch editing of tasks, which is a feature that is available in many other task managers in one form or another. It also does not support location-based notifications, which would have been useful to send reminders to users based on their location. The application does not make use of advancements in machine learning: learning users’ preferences over time to provide helpful shortcuts when creating new tasks could have been a fantastic time saver. ProductiviDo could have stored task templates such that creating recurring tasks would be faster and more intuitive by providing logical default values based on what the algorithm thinks is going to be needed next. Similarly, the app could add locations, people, and other properties to tasks once it figures out that certain actions only happen in given locations or with the same people repeatedly. Yet another feature that would have proven handy would be to provide time-based notifications based on predictions from past data derived from tracking user movements and time taken to complete specific types of tasks. For instance, if a user has a task that is due at 9:00 AM, the app could send a notification to the user’s phone at 8:45 AM if the user is at home, but at 8:30 AM if the user is at work once the travel time could reliably be estimated from prior activities.

6 Conclusion

6.1 Salient takeaways

ProductiviDo set out to fathom the needs of the modern-day, demanding user and to provide a novel solution to the problem of task management and scheduling. The application was designed from the start to work alongside Google Calendar, which is a popular calendar application that is used by many millions of people around the world. The application was designed to be a simple and intuitive task manager that would allow users to create tasks and events, to schedule them, and to track their progress. While there is room for improvement, ProductiviDo goes far beyond a proof of concept or “Goldilocks Quality” and shows how it can be used by anyone who wants to manage their tasks and schedule more efficiently, as it relies on strong scientific foundations that were dispersed throughout this work as convincing evidence backing up such a claim.

One of the key takeaways from this work is that despite a crowded task management digital market, innovation can take place by taking inspiration from different fields and by applying them to the problem at hand. In the case of ProductiviDo, the needs of business users and productivity seekers alike were distilled into an almost mechanical process, where the application flow is extremely simple and strives to put forward in a visually sensible manner all the required functionality to get work done without friction: this meant taking a few assumptions about users for granted, knowingly reducing the user base to better serve those who will resonate with the offering.

There are a few dominant players in the industry, albeit niches are still well underserved. Almost every software application out there in this space endeavors to serve customers in a very generic way, but it could be flourishingly argued that a task manager could be designed to serve a specific industry. For instance, a task manager for a construction company could be designed to be more intuitive for construction workers, who are not necessarily computer savvy, and could be designed to be more efficient for project managers, who are likely to be more tech-savvy. The same could be said for a task manager for a law firm, a task manager for a restaurant, or a task manager for a school. The possibilities are endless, and the key is to understand the needs of the users and to design the application to be as intuitive as possible. Even though “task templates” can, up to a point, be used to achieve this, nothing competes with a well-designed application that is tailored to the needs of a well-known target audience. Such applications will by definition be adopted within a given niche and will dispense unique value to the users. A generic task manager may be suitable for simple projects and may require fiddling around with the settings to get it to work for a specific use case, but a task manager dedicated to culinary chefs will natively understand and care for concepts such as “ingredients” and “recipes”, which will be baked into the product (no pun intended).

6.2 Improvements and future research

Even though Google users abound, using exclusively Google Calendar as an calendar integration is not a universal solution. In the future, it would be interesting to explore the possibility of integrating other calendar applications such as Apple Calendar, Outlook, and others. This would allow users to use the application with their preferred calendar application and would also allow for a more seamless integration with the calendar application, which would be a great benefit to users.

Expanding on the notion of “delegatees” explored in this project, a task manager with built-in collaboration would be welcome to further satisfy the needs of business users. This is because a calendar can become social and collaborative by itself, and it is not uncommon for colleagues or friends to share their calendars with others. This is especially true for business users, who often need to coordinate their activities with other members of their team. It would certainly not be far-fetched to have business teams be formed and given access to their shared task assignments and calendars. This would allow for a more efficient and transparent way of working together, and it would also allow for a more efficient way of tracking the progress of a project and would pave the way to generating insightful analytics dashboards, another place where machine learning would shine by finding patterns and making it possible to optimize overlapping schedules and projects based on people’s availability, area of expertise and so on.

Recent reports have shown that job creation and entrepreneurship is at its highest peak and that the United States of America is leading the way in terms of new business creation, with literally millions of companies being born each year. This is a great sign for the economy and it shows numerous opportunities in the business world when considering task managers embarking on a journey to become a collaborative platform for teams and businesses within a specific line of work. In the meantime, ProductiviDo comes to the rescue of the millions of people who are looking for simplicity and intuitiveness in a task manager to get their work done with less technological grinding.

Demo of the app

Book summary: Effective Java

Sébastien Lavoie — Sun, 11 Jun 2023 18:15:00 -0600

Book summary

Creating and destroying objects

Item 1: Consider static factory methods instead of constructors

Use static factory methods to increase readability and make classes more flexible (e.g., Foo.withOpenBar() instead of new Foo(true)).

Item 2: Consider a builder when faced with many constructor parameters

When a class has many parameters, use a builder instead of a “telescoping constructor pattern” (i.e., having a separate method to call the class with any number of parameters, which may or may not all be used, leading to confusion and difficulty to use).
The “JavaBeans pattern” uses setters and a parameter-less constructor, allowing the use of any parameters as needed when using the code (e.g., Foo.setMethod()). This allows the class to enter in an inconsistent state (it is difficult to guarantee that a combination of used parameters makes sense) and makes it difficult to create an immutable class.
This pattern allows calling code like NutritionFacts cocaCola = new NutritionFacts.Builder(240, 8).calories(100).sodium(35).carbohydrate(27).build();, which basically instantiate a class with optional parameters as would be the case in Python.

Item 3: Enforce the singleton property with a private constructor or an enum type

The recommended path forward is to use an enum type like so:

 java
 
 
 
public enum MyClass {
 INSTANCE;
 public void myMethod() { ... }
 }

...

MyClass instance = MyClass.INSTANCE;
instance.myMethod();

Item 4: Enforce non-instantiability with a private constructor

Simply define the class constructor to be private so that the class cannot be instantiated. This is useful for utility classes that may contain a bunch of static methods where instantiating the class itself is nonsensical.

Item 5: Prefer dependency injection to hardwiring resources

The example given with a SpellChecker that depends on a Dictionary makes a lot of sense: instead of having a class SpellChecker define a Dictionary that won’t be reusable elsewhere or that may need to change, pass the Dictionary into the constructor of the class so that different dictionaries can be used with the same class, making it easier to test things independently and update an existing implementation.

Item 6: Avoid creating unnecessary objects

The author provides two striking examples of where creating unnecessary objects can be detrimental to performance: using Long sum = 0L; instead of long sum = 0L; when summing up positive integers and matching a regular expression on a string repeatedly instead of compiling the regular expression once for re-use. In short, avoid creating unnecessary objects and prefer primitives over boxed primitives.

Item 7: Eliminate obsolete object references

The author provides an example of a stack implementation that has a memory leak because the stack maintains obsolete references to objects that have been popped off the stack. The solution is to null out references once they are no longer needed, which in this case happened in a class where manual memory management was occurring. This is a problem because the garbage collector has no way of knowing that the objects it references after a certain portion of the array are no longer needed.
The author also shows an example of a cache that uses a WeakHashMap to store the cache, which allows the garbage collector to remove entries from the cache when they are no longer referenced elsewhere in the program.

Item 8: Avoid finalizers and cleaners

“Finalizers are unpredictable, often dangerous, and generally unnecessary.” They can lead to performance issues and resource leaks. They should be avoided. There is no guarantee that they will be called in a timely manner, or at all.

Item 9: Prefer try-with-resources to try-finally

The author provides an example of a class that reads the first line of a file and returns it as a string. The class uses a BufferedReader to read the file and a FileReader to open the file. The problem is that the FileReader is never closed, which can lead to resource leaks. The solution is to use a try-with-resources block, which will automatically close the FileReader when the block is exited.

Methods common to all objects

Item 10: Obey the general contract when overriding `equals`

The equals method must be reflexive, symmetric, transitive, consistent, and x.equals(null) must return false.
It is generally a bad idea to @Override the equals method when inheriting from a concrete class, as it is difficult to ensure that the contract is respected. It is better to use composition instead of inheritance in this case.

Item 11: Always override `hashCode` when you override `equals`

If two objects are equal according to the equals method, they must have the same hash code. The reverse is not true: two objects with the same hash code are not necessarily equal.
Using the AutoValue framework in Java, it is possible to automatically generate equals and hashCode methods for a class.

Item 12: Always override `toString`

The toString method should return a concise, useful description of the object. It should be overridden in every class unless the class is a utility class or a small value class.
The toString method should return all of the interesting information contained in the object when it is practical to do so. It should not return the memory address of the object.

Item 13: Override `clone` judiciously

The clone method is broken in Java. It is better to use a copy constructor or a copy factory instead of the clone method.
One exception is when a class implements the Cloneable interface and has a final field that is a reference to an array or a final field that is a reference to an object. In this case, the clone method can be used to create a copy of the object.

Item 14: Consider implementing `Comparable`

Implement the Comparable interface when a class has a natural ordering. This allows the class to be used in sorted collections and provides a way to compare objects of the class.
An example is given to compare phone numbers using the Comparable interface, where chaining happens in order to compare from the most significant field to the least significant field, e.g.:

 java
 
 
 
// Comparable with comparator construction methods
 private static final Comparator<PhoneNumber> COMPARATOR =
 comparingInt((PhoneNumber pn) -> pn.areaCode)
 .thenComparingInt(pn -> pn.prefix)
 .thenComparingInt(pn -> pn.lineNum);
 public int compareTo(PhoneNumber pn) {
 return COMPARATOR.compare(this, pn);
}

Classes and interfaces

Item 15: Minimize the accessibility of classes and members

Make each class or member as inaccessible as possible. This allows for better encapsulation and makes it easier to change the implementation of a class without breaking the code that uses it.
In public classes, instance fields should rarely be public, as that generally makes the classes themselves non-thread-safe. Instead, use accessor methods to access the fields.
Static final fields can be made public if they are immutable.

Item 16: In public classes, use accessor methods, not public fields

When a class needs to be accessed outside of its package, provide accessor methods instead of making the fields public. This allows the class to change its implementation without breaking the code that uses it.
This does not really apply to nested or private classes that are only used within the package as long as they still represent data structures usefully, as this leads to less visual clutter.

Item 17: Minimize mutability

Don’t allow the class to use mutators.
Prevent the class from being extendable.
Make all fields final.
Make all fields private.
Don’t write setters when they are not needed.
Reducing the number of states a class can find itself in makes it easier to reason about the class and reduces the number of bugs that can occur.

Item 18: Favor composition over inheritance

Inheritance is often overused and can lead to fragile code. It is better to use composition instead of inheritance where appropriate.
When a class takes a reference to another class as a parameter in its constructor while extending the interface for the class it is taking as a parameter via a forwarding class, it is using composition, as it calls super(otherClass) in its constructor. This allows the class passed as a parameter to be extended without having to change the class itself. In this way, the class using composition becomes a wrapper that can add functionality on top of the class it is wrapping. The books gives an example of a ForwardingSet that extends Set and takes a Set as a parameter in its constructor. The ForwardingSet can then add functionality on top of the Set it is wrapping, in this case by counting the number of times the add method is called.
The main, appropriate use case for inheritance is when a class is a subtype of another class and satisfies the is-a relationship.

Item 19: Design and document for inheritance or else prohibit it

To test a class for inheritance, try to extend it. If it is not possible to extend the class, it is not designed for inheritance.
No overridable methods should be called in the constructor, because the superclass runs before the subclass, and the subclass will not have been initialized yet.
It is better to prohibit inheritance by declaring a class to be final and ensuring that no constructors are accessible.

Item 20: Prefer interfaces to abstract classes

Interfaces are better than abstract classes because they allow for multiple inheritance, they allow for the creation of mixins, and they allow for the creation of tag interfaces.
Interfaces can define types that do not need to be represented hierarchically. For instance, a DirectorPhotographer interface can be created that extends both the Director and Photographer interfaces.

Item 21: Design interfaces for posterity

Multiple programmers should implement interfaces in different ways, and multiple client programs should use the interfaces to ensure they satisfy all intended uses. Flaws in interfaces are easier to correct before release.
The moral is to be cautious when adding methods to existing interfaces using default methods and to thoroughly test and design interfaces to avoid issues and flaws in the long term.

Item 22: Use interfaces only to define types

Interfaces serve as types that define what a client can do with instances of a class that implements the interface.
Constant interfaces consist solely of static final fields, exporting constants. They are a poor use of interfaces as they expose implementation details and confuse users.
- Implementing a constant interface leaks implementation details into the class’s API and creates a commitment to maintain binary compatibility. Subclasses of a class implementing a constant interface are also affected.
- If constants are tied to an existing class or interface, add them directly to that class or interface. If constants are best viewed as members of an enumerated type, use an enum type. Otherwise, use a noninstantiable utility class to export the constants.
Interfaces should be designed to define types and not used solely for the purpose of exporting constants.

Item 23: Prefer class hierarchies to tagged classes

Tagged classes are verbose, error-prone, and inefficient. They have boilerplate code, mix multiple implementations in a single class, increase memory footprint, require careful initialization, and make it difficult to add new flavors.
- Instead, abstract classes can be used as the root of the hierarchy, with concrete subclasses representing each flavor.
Class hierarchies eliminate the shortcomings of tagged classes. They are simpler, clearer, have no boilerplate, ensure field initialization, prevent missing cases, support independent extensibility, and provide distinct data types for each flavor.
- Class hierarchies can reflect natural relationships among types. For example, a square can be represented as a subclass of a rectangle in the hierarchy.

Item 24: Favor static member classes over nonstatic

There are four kinds of nested classes in Java: static member classes, nonstatic member classes (inner classes), anonymous classes, and local classes.
- Static member classes are ordinary classes declared inside another class and have access to all members of the enclosing class. They are static members of their enclosing class and serve as public helper classes. They obey the same accessibility rules as other static members.
- Nonstatic member classes are implicitly associated with an instance of the enclosing class and can access methods and fields of the enclosing instance. They require an enclosing instance to be created and cannot exist independently.
- Private static member classes are used to represent components of the object represented by their enclosing class. They don’t require access to the enclosing instance and avoid unnecessary memory usage compared to nonstatic member classes.
If a member class doesn’t require access to an enclosing instance, it should be declared as a static member class to avoid the overhead of an extra reference and potential memory leaks.
Static member classes are preferred over nonstatic member classes unless access to the enclosing instance is necessary.

Item 25: Limit source files to a single top-level class

The risks stem from the possibility of providing multiple definitions for a class, and the order in which source files are passed to the compiler determines which definition is used.
If there is a need to group related classes, consider using static member classes as an alternative. Static member classes enhance readability and allow for reduced accessibility by declaring them private.

Generics

Item 26: Don’t use raw types

Raw types are generic types used without specifying any type parameters.
Using raw types can lead to errors at runtime (the compiler doesn’t provide type safety) and should be avoided.
Instead of using raw types, it is recommended to use parameterized types to ensure type safety and expressiveness.
Unbounded wildcard types (e.g., Set<?>) can be used when the actual type parameter is unknown or doesn’t matter: they offer flexibility while still maintaining type safety.

Item 27: Eliminate unchecked warnings

Some unchecked warnings are easy to eliminate by specifying the type parameter or using the diamond operator (<>).
Treat unchecked warnings seriously and make efforts to eliminate them, ensuring type safety and reducing the potential for ClassCastException at runtime.

Item 28: Prefer lists to arrays

Arrays are covariant, which means if Sub is a subtype of Super, then the array type Sub[] is a subtype of the array type Super[]. Generics, on the other hand, are invariant.
Arrays are reified, meaning they enforce their element type at runtime, while generics use erasure, enforcing their type constraints only at compile time.
Arrays do not provide compile-time type safety, and errors may only be discovered at runtime, which is undesirable.
Generic collections should be used instead of arrays to ensure better type safety and interoperability.
Using lists instead of arrays allows for better compile-time type safety, even if it sacrifices some conciseness or performance.

Item 29: Favor generic types

The use of generic types eliminates the need for explicit casts and ensures type safety at compile time.
Bounded type parameters can be used to restrict the permissible values of type parameters in generic types, allowing for more specific behavior without explicit casting.
It is recommended to use generic types whenever possible to avoid casting and improve type safety. Existing non-generic types should be generified, making them easier to use without breaking existing clients.

Item 30: Favor generic methods

To write a generic method, add type parameters to its declaration and use those type parameters throughout the method.
Generic methods provide type safety and ease of use by eliminating the need for explicit casts.
Bounded wildcard types can be used in generic methods to make them more flexible and allow for different types of arguments.

Item 31: Use bounded wildcards to increase API flexibility

Parameterized types are invariant, meaning that List<String> is not a subtype of List<Object>.
Wildcard types allow you to specify that a parameter can be a subtype of a certain type, using ? extends Type for producers and ? super Type for consumers.
Wildcard types should not be used as return types.
Wildcard types are particularly useful when working with comparables and comparators.

Item 32: Combine generics and `varargs` judiciously

Varargs methods and generics do not interact well due to the nature of varargs creating arrays to hold the arguments.
Alternatively, using a List parameter instead of a generic varargs parameter can provide type safety, relying on the List.of method to handle a variable number of arguments. The code may be slightly more verbose and slower.

Item 33: Consider typesafe heterogeneous containers

Common uses of generics involve parameterizing the container itself, such as Set<E> and Map<K,V>, which limits the number of type parameters per container.
Runtime type safety can be achieved by checking the type relationship between the key and the instance being stored using a dynamic cast.
Bounded type tokens can be used to restrict the types that can be passed as keys by using a bounded type parameter or a bounded wildcard.

Enums and annotations

Item 34: Use enums instead of `int` constants

Some variants use String constants instead of int constants, but this approach has even more disadvantages. It can lead to errors, performance issues, and lacks type safety.
Enum types in Java are full-fledged classes and offer type safety, expressive power, and additional benefits.
Enum types consist of a fixed set of constants, each represented as an instance of the enum class. Enum constants are exported via public static final fields. Enum types are effectively final and can’t be extended or instantiated outside the declared constants.
Enum types provide compile-time type safety. If you declare a parameter or variable of an enum type, the compiler ensures that only valid enum constants can be assigned or passed as arguments.
Each enum type has its own namespace, allowing identically named constants in different enum types without conflicts.
Enum types can have methods, fields, and can implement interfaces.
Enum types have comparable performance to int constants. The space and time cost of loading and initializing enum types is usually negligible.

Item 35: Use instance fields instead of ordinals

While it’s possible to derive associated int values from the ordinal, it is strongly discouraged. Using ordinal-based calculations for associated values can lead to maintenance issues and limitations.
The recommended solution is to store associated int values in instance fields instead of deriving them from the ordinal. By assigning each enum constant a specific value in the constructor, you can avoid the pitfalls of relying on the ordinal.

Item 36: Use `EnumSet` instead of bit fields

Traditionally, if the elements of an enumerated type are used primarily in sets, the int enum pattern (Item 34) is used. Each constant is assigned a different power of 2, allowing bitwise OR operations to combine multiple constants into a set.
- It is harder to interpret a bit field when printed as a number compared to a simple int enum constant.
- There is no easy way to iterate over all the elements represented by a bit field.
- Bit fields require choosing a specific type (int or long) with a fixed width, which limits the maximum number of bits that can be used without changing the API.
EnumSet provides type safety and interoperability, just like any other set implementation.
- Built-in operations: Bulk operations like removeAll and retainAll are implemented efficiently using bitwise arithmetic.

Item 37: Use `EnumMap` instead of ordinal indexing

Ordinal indexing involves using an enum’s ordinal value as an index to access elements in an array or list. Problems:
- Using ordinals directly lacks the type safety provided by enums.
- Due to the incompatibility of arrays with generics, the program requires an unchecked cast, resulting in a compile-time warning.
- If the ordinal values change, the code breaks, and there’s no compile-time checking to ensure correct indexing.
- When using a two-dimensional array indexed by ordinals, the size of the table grows quadratically, even if some entries are null.
The EnumMap is a specialized map implementation designed to work efficiently with enum keys.
- They provide type safety, eliminating the need for unchecked casts. They also allow for cleaner code with no manual labeling of output.

Item 38: Emulate extensible enums with interfaces

Enum types in Java are not extensible by design.
The basic idea is to define an interface for the opcode type and an enum that implements this interface.
Emulating extensible enums with interfaces provides flexibility but has limitations. Implementations cannot be inherited from one enum type to another, and some code duplication may occur. Shared functionality can be encapsulated in a helper class or static helper method to reduce duplication.

Item 39: Prefer annotations to naming patterns

Annotations offer a superior alternative to naming patterns. They address the shortcomings of naming patterns by providing compile-time checking, better enforcement of usage constraints, and support for associating parameter values with program elements (e.g., @Test).
While most programmers may not need to define their own annotation types, they should use predefined annotations provided by Java and consider using annotations provided by IDEs or static analysis tools. These annotations can enhance code quality and diagnostic information.

Item 40: Consistently use the `Override` annotation

The @Override annotation is used on method declarations to indicate that the annotated method overrides a declaration in a supertype (class or interface).
Consistently using @Override protects against bugs by ensuring that overridden methods are correctly implemented. It helps catch errors where a method is intended to override a superclass method but ends up overloading it instead.
In abstract classes and interfaces, it is worth annotating all methods that are believed to override superclass or super-interface methods, regardless of their nature (concrete or abstract). This helps ensure correctness and prevents accidental additions of new methods.

Item 41: Use marker interfaces to define types

A marker interface is an interface that contains no method declarations but serves as a marker to designate a class implementing it as having a specific property. An example is the Serializable interface, which marks a class as serializable.
Marker interfaces define a type that is implemented by instances of the marked class, allowing compile-time error detection. They provide stronger type checking compared to marker annotations, which cannot define a type.
If the marker applies only to classes and interfaces, and there is a possibility of writing methods that accept only objects with the marking, a marker interface should be used. This enables compile-time type checking. If there is no need for such methods or if the marker is part of a framework heavily using annotations, a marker annotation is more appropriate.

Lambdas and streams

Item 42: Prefer lambdas to anonymous classes

Lambdas are concise expressions that can be used to create instances of functional interfaces, providing a more compact alternative to anonymous classes.
Lambdas lack names and documentation, making them less suitable for complex computations or those exceeding a few lines.
Anonymous classes are still necessary for creating instances of abstract classes or interfaces with multiple abstract methods. Lambdas cannot obtain a reference to themselves, unlike anonymous classes.

Item 43: Prefer method references to lambdas

Method references offer a more concise alternative to lambdas. They allow you to refer to a method by name instead of providing a lambda expression. This is particularly useful when the lambda expression only calls a method without any additional logic. E.g.:
- Method reference: map.merge(key, 1, Integer::sum);
- Lambda expression: service.execute(() -> action());
By using method references, you can eliminate the need to declare parameter names explicitly in the lambda expression.

Item 44: Favor the use of standard functional interfaces

Instead of using the Template Method pattern, which involves sub-classing and overriding methods, the modern approach is to provide static factories or constructors that accept function objects as parameters.
The java.util.function package provides a collection of standard functional interfaces for various use cases. When choosing a functional parameter type, it is recommended to use the standard functional interfaces instead of creating custom interfaces.
Functional interfaces should be annotated with the @FunctionalInterface annotation. This annotation serves as documentation, enforces the single abstract method requirement, and prevents accidental addition of abstract methods in the interface.
In API design, it’s important to avoid overloading methods that take different functional interfaces in the same argument position. This can create ambiguity for clients, and it’s best to design the API to prevent this situation.

Item 45: Use streams judiciously

Streams were introduced in Java 8 as a way to perform bulk operations on sequences of data elements. A stream represents a sequence of elements, and a stream pipeline consists of a source stream, intermediate operations, and a terminal operation.
The streams API provides a fluent API, allowing multiple operations to be chained together in a single expression. This enables concise and readable code.
Overusing streams can make code harder to read and maintain.
They have restrictions on accessing and modifying local variables and limited control flow capabilities.
Streams are well-suited for tasks such as transforming elements, filtering, combining elements, accumulating into collections, and searching. If a computation requires access to corresponding elements from multiple stages of the pipeline, streams might not be the best choice.

Item 46: Prefer side-effect-free functions in streams

The fundamental aspect of the streams paradigm is structuring computations as a sequence of transformations where each stage’s result is a pure function of the previous stage’s result. A pure function depends only on its input and does not have any side effects or modify any state.
When using stream operations, both intermediate and terminal, it’s important to ensure that the function objects passed to them are free of side effects. This means they should not depend on mutable state or update any state.
The correct usage of streams involves utilizing collectors, which encapsulate reduction strategies. Collectors can be used to gather elements into collections or create maps based on grouping or other criteria (e.g., toList, toSet, toMap, groupingBy).
There are parallel and concurrent variants of collectors, such as groupingByConcurrent and toConcurrentMap, which efficiently handle parallel execution and produce ConcurrentHashMap instances.

Item 47: Prefer Collection to Stream as a return type

A for loop cannot be used to iterate over a stream, and it’s not possible to add elements to a stream. This makes streams less flexible than collections.
- To enable iteration over a stream with a for-each loop, an adapter method can be created. The method converts a Stream to an Iterable, allowing for the use of for-each loops.
When designing a public API that returns a sequence, it is important to consider users who may prefer stream pipelines or for-each loops. The Collection interface, being a subtype of Iterable and providing a stream method, is often the best choice for a return type. Arrays can also be used when appropriate.

Item 48: Use caution when making streams parallel

Simply adding the parallel method to a stream pipeline does not guarantee improved performance. In some cases, it can lead to liveness failures, incorrect results, and unpredictable behavior.
Performance gains from parallelism are most significant when working with data structures like ArrayList, HashMap, HashSet, ConcurrentHashMap, arrays, int ranges, and long ranges. These data structures can be easily split into sub-ranges, enabling efficient work division among parallel threads.
Parallelization should only be used as a performance optimization.
Certain domains, such as machine learning and data processing, are well-suited for parallel speedups.

Methods

Item 49: Check parameters for validity

Clearly document the restrictions on parameter values in the method’s documentation. Enforce these restrictions by performing parameter checks at the beginning of the method body. Detecting errors as soon as possible is crucial for maintaining code integrity.
If a method fails to check its parameters, it may result in confusing exceptions, incorrect results, or compromised object states.
For public and protected methods, use the @throws tag in Javadoc to document the exception that will be thrown if a parameter value violates a restriction.
Use the Objects.requireNonNull method, introduced in Java 7, to check for null values. It is flexible and convenient, allowing you to specify your own exception detail message if desired. It returns the non-null value, enabling simultaneous null check and usage.
Nonpublic methods can use assertions to check parameters since the package author controls the method’s usage. Assertions throw AssertionError if the condition fails. Enabling assertions with the -ea flag is necessary for assertions to take effect.
While it’s important to check parameter validity, avoid imposing unnecessary restrictions on parameters. Design methods to be as general as possible, accepting a wide range of valid parameter values.

Item 50: Make defensive copies when needed

To protect against ill-behaved clients or honest mistakes, it is crucial to write robust classes that can handle unexpected behavior.
Classes that contain mutable objects can be vulnerable to attacks that exploit their mutability.
make defensive copies of each mutable parameter in the constructor. Use these copies as components of the class instance instead of the original objects. This ensures that changes to the original objects do not affect the class’s internal state.
Accessor methods that provide access to mutable internal fields should also return defensive copies of the fields.
Whenever possible, use immutable objects as components of your classes to eliminate the need for defensive copying. Immutable objects are inherently safe and can simplify your code.

Item 51: Design method signatures carefully

Choose method names carefully.
Avoid excessive convenience methods. Convenience methods are useful, but they can make the API more difficult to learn and maintain.
Limit parameter list length.
Prefer interfaces over classes for parameter types. Use interfaces to define parameters instead of specific classes whenever possible. This allows for more flexibility and enables the use of different implementations. For example, use the Map interface instead of the HashMap class to allow different Map implementations to be used.
Use enums instead of boolean parameters. Enums allow for future expansion by adding more options without changing method signatures.

Item 52: Use overloading judiciously

Overloading can lead to unexpected behavior: If overloaded methods have the same number of parameters and the same compile-time type of the arguments, the selection of the method is ambiguous.
Overloaded methods should have “radically different” parameter types.
Generics can create situations where two overloadings with different functional interfaces in the same argument position cause confusion. It’s recommended not to overload methods that take different functional interfaces in the same argument position.

Item 53: Use varargs judiciously

Varargs allow methods to accept a variable number of arguments of a specified type. They work by creating an array internally and passing the array to the method.
Use varargs when you want a method with a variable number of arguments. For example, when calculating the sum of integers or finding the minimum of a set of integers.
Declare the method to take two parameters: one normal parameter and one varargs parameter. This ensures that the method can handle both cases when at least one argument is passed and when more than one argument is passed.
Be aware that every invocation of a varargs method causes an array allocation and initialization, so consider the performance implications in performance-critical situations.
- To optimize performance while still using varargs, you can provide overloaded methods with a fixed number of parameters and a single varargs method for cases when the number of arguments exceeds a certain threshold.

Item 54: Return empty collections or arrays, not nulls

Returning null to indicate an empty collection or array is unnecessary and error-prone. It requires extra code in the client to handle the null return value and increases the risk of null pointer errors.
Instead of returning null, return empty collections or arrays. This can be achieved by creating a new instance of the appropriate collection or array type. For example, returning new ArrayList<>(cheesesInStock) will return an empty ArrayList if cheesesInStock is empty.

Item 55: Return optionals judiciously

Exceptions should be reserved for exceptional conditions, and returning null requires special-case code to handle it and increases the risk of null pointer errors.
The Optional<T> class represents an immutable container that can hold either a single non-null value of type T or nothing at all (empty). It provides a more flexible and less error-prone alternative to exceptions or null returns.
Avoid using Optional<T> for container types like collections, maps, streams, arrays, and other optionals. Return the empty container directly instead of wrapping it in an optional.

Item 56: Write doc comments for all exposed API elements

Javadoc is a utility that generates API documentation automatically from specially formatted doc comments in the source code.
Precede every exported class, interface, constructor, method, and field declaration with a doc comment. It describes the contract between the API element and its client.
Document the preconditions, post-conditions, and side effects of methods. Use the @param tag to describe parameters, @return tag for the return value (if not void), and @throws tag for exceptions thrown.
Use HTML tags in doc comments to format and structure the generated documentation.
Use {@code} tag to render code fragments in code font and to suppress processing of HTML markup and nested Javadoc tags within the code fragment.
Write summary descriptions as verb phrases for methods and constructors, and noun phrases for classes, interfaces, and fields. The first sentence becomes the summary description.
Use {@literal} tag to include HTML metacharacters in documentation, such as <, >, and &.
Document self-use patterns in classes designed for inheritance using the @implSpec tag. It describes the contract between the method and its subclass.
Package-level doc comments should be placed in package-info.java file. Similarly, module-level comments should be placed in module-info.java file.
Document thread-safety and serializability of classes.
Read the generated documentation to ensure clarity and make any necessary improvements to the doc comments.

General programming

Item 57: Minimize the scope of local variables

Declare variables where they are first used, rather than at the beginning of a block. This approach eliminates clutter and helps readers understand the purpose and type of the variable when it is used.
Avoid prematurely declaring variables outside of the block in which they are used. By doing so, you ensure that the variable’s scope begins at the appropriate point and ends when it is no longer needed.
Take advantage of the scope-limiting capabilities of for loops.
Prefer for loops over while loops when the loop variable is not needed after the loop terminates.
Minimizing the scope of local variables helps prevent copy-and-paste errors.
Consider using multiple loop variables within a for loop to avoid redundant computations. This approach can improve performance by storing the limit of the first variable in a second variable, eliminating the need for redundant computations in each iteration.
Keep methods small and focused to minimize the scope of local variables.

Item 58: Prefer `for-each` loops to traditional `for` loops

Traditional for loops for iteration over collections and arrays can be cluttered and prone to errors. They require explicit use of an iterator or an index variable.
The for-each loop (enhanced for statement) solves these issues by eliminating the need for explicit iterators or index variables: for (Element e : elements) { ... }
For-each loops can be used with both collections and arrays, making it easy to switch the implementation type of a container without changing the loop syntax.
For nested iteration, for-each loops offer greater clarity and simplicity compared to traditional for loops. They avoid common bugs that can occur when using multiple iterators or index variables.
There are certain situations where for-each loops cannot be used, such as when performing destructive filtering, transforming values, or iterating in parallel. In these cases, traditional for loops or other approaches may be necessary.
The Iterable interface allows objects to be iterated over using the for-each loop. By implementing the Iterable interface and providing an iterator method, your custom types can be used with for-each loops.

Item 59: Know and use the libraries

Libraries, such as Random, offer well-designed and thoroughly tested methods for common tasks.
The use of libraries saves time and effort. You can focus on your application logic instead of spending time on low-level implementations.
Standard libraries tend to improve over time in terms of performance and functionality.
While the libraries are extensive, every programmer should be familiar with the basics of java.lang, java.util, java.io, and their sub-packages. Additional knowledge can be acquired on an as-needed basis.
Certain libraries, such as the collections framework, streams library, and concurrency utilities in java.util.concurrent, should be part of every programmer’s toolkit.
Avoid reinventing the wheel. If a common task can be accomplished using existing library facilities, utilize them instead of writing your own code. Library code is often more robust and undergoes more scrutiny than individual developers can afford.

Item 60: Avoid float and double if exact answers are required

float and double types are designed for scientific and engineering calculations and provide fast but approximate results using binary floating-point arithmetic.
The inherent imprecision of float and double types makes them unsuitable for applications where exact results are necessary, such as monetary calculations.
Performing arithmetic operations with float and double types can lead to unexpected rounding errors and inaccuracies.
Rounding the results before printing is not a reliable solution to the imprecision issue.
To achieve accurate results in monetary calculations, it is recommended to use alternative data types like BigDecimal, int, or long.
BigDecimal is a precise data type that can handle exact decimal calculations. It is suitable for situations where accuracy is paramount, such as monetary calculations. However, it is less convenient to use than primitive types and slower in performance.
An alternative approach is to perform calculations in cents using int or long types, keeping track of the decimal point manually. This approach is more efficient and convenient than using BigDecimal but has limitations in handling large quantities.
Use BigDecimal when precision is critical, int or long when performance and convenience are important, and float or double when approximate results are acceptable.
BigDecimal offers full control over rounding, which is useful in business calculations with legally mandated rounding behavior.

Item 61: Prefer primitive types to boxed primitives

Java has two types: primitives (e.g., int, double, boolean) and reference types (e.g., String, List). Each primitive type has a corresponding boxed primitive type (e.g., Integer, Double, Boolean).
Primitives only have values, while boxed primitives have both values and identities.
Boxed primitives have an additional nonfunctional value: null.
Primitives are more time- and space-efficient compared to boxed primitives.
Comparing boxed primitives with the == operator performs an identity comparison, not a value comparison.
Auto-unboxing a null boxed primitive results in a NullPointerException.
Auto-boxing and auto-unboxing operations can lead to unnecessary object creations and performance issues.
Use primitives whenever possible, especially when comparing values or performing computations.
Boxed primitives are necessary in specific situations:
- When using collections, as primitives cannot be directly added to collections;
- As type parameters in parameterized types and methods;
- When making reflective method invocations.

Item 62: Avoid strings where other types are more appropriate

Strings should be used to represent text and should not be used as substitutes for other value types.
Avoid using strings to represent numeric data. Translate numeric data into the appropriate numeric type, such as int, float, or BigInteger, to ensure accuracy and enable numerical operations.
Enum types are better suited for enumerated type constants than strings. Use enums to represent a fixed set of values with distinct identities and behaviors.

Item 63: Beware the performance of string concatenation

The string concatenation operator is convenient for combining a few strings but does not scale well. When using the concatenation operator repeatedly, the time required is quadratic in the number of strings being concatenated because the contents of both strings are copied each time.
To achieve better performance, use a StringBuilder instead of the string concatenation operator when constructing a string from multiple components. StringBuilder provides efficient appending of strings, and its performance is linear in the number of strings being concatenated.
The StringBuilder approach can significantly outperform the string concatenation operator, especially when the number of strings or the size of the resulting string is large. Preallocating a StringBuilder with an appropriate capacity further improves performance by avoiding automatic growth.
Consider alternatives to string concatenation, such as using a character array or processing strings individually instead of combining them if performance is a concern.

Item 64: Refer to objects by their interfaces

Favor using interfaces as types over classes when referring to objects. Parameters, return values, variables, and fields should all be declared using interface types if appropriate interfaces exist.
By using interfaces as types, your code becomes more flexible. If you decide to switch implementations, you can simply change the class name in the constructor (or use a different static factory) without affecting the surrounding code.
It is appropriate to refer to objects by their class rather than an interface when no appropriate interface exists, such as with value classes like String or BigInteger.
Classes implementing an interface but providing additional methods not found in the interface should be referred to by their class only if the program relies on those extra methods.

Item 65: Prefer interfaces to reflection

Reflection provides programmatic access to arbitrary classes and allows manipulation of their constructors, methods, and fields. However, it comes with several drawbacks:
- Loss of compile-time type checking and exception checking;
- Clumsy and verbose code required for reflective access;
- Poor performance compared to normal method invocation.
Reflection is rarely needed in most applications.

Item 66: Use native methods judiciously

The use of native methods has serious disadvantages:
- Native languages are not memory safe, so applications using native methods are susceptible to memory corruption errors;
- Native methods make programs less portable as they are more platform-dependent than Java;
- Debugging native code is more challenging compared to Java code;
- Garbage collection and memory management become more complex since the garbage collector cannot track or automate native memory usage;
- Native methods require writing and maintaining “glue code,” which can be difficult to read and tedious to write.
Use as little native code as possible.
Thoroughly test the native code to avoid bugs that could corrupt the entire application.

Item 67: Optimize judiciously

Premature optimization is often counterproductive. More computing sins are committed in the name of efficiency than for any other reason. Optimization without necessarily achieving it can lead to software that is neither fast nor correct and can be difficult to fix.
Good programs should prioritize sound architectural principles over performance. Write good programs first, and if necessary, optimize them later. Good programs are designed with information hiding, allowing individual components to be changed without affecting the rest of the system.
Performance should be considered during the design process. Avoid design decisions that limit performance, especially in APIs, wire-level protocols, and persistent data formats. API design can have a significant impact on performance, so be mindful of decisions such as mutability, inheritance, and interface usage.
API design and architectural decisions can have real performance consequences. Consider the trade-offs and implications of your design choices. For example, returning mutable objects may result in unnecessary defensive copying, and using implementation types instead of interfaces can limit future performance improvements.
Before optimizing, ensure that your program has a clear, concise, and well-structured implementation. Measure performance before and after each attempted optimization. Profiling tools can help identify performance bottlenecks and guide optimization efforts. Be aware that Java’s performance model is less well-defined compared to lower-level languages, making it essential to measure the effects of optimizations on different implementations and hardware platforms.
Algorithmic changes should be prioritized over low-level optimizations. If a quadratic or inefficient algorithm exists, no amount of tuning will fix the underlying problem.
Performance measurement and optimization are iterative processes. Measure performance, identify bottlenecks, optimize relevant parts, and repeat until satisfied.

Item 68: Adhere to generally accepted naming conventions

Typographical Naming Conventions:
- Package and module names should be hierarchical, separated by periods, and consist of lowercase alphabetic characters and digits. The name should begin with the organization’s Internet domain name, with the components reversed;
- Class and interface names should consist of one or more words, with the first letter of each word capitalized. Abbreviations should be avoided, except for acronyms and common abbreviations.
- Method and field names should follow the same typographical conventions as class and interface names, with the first letter lowercase for methods and fields. Acronyms occurring as the first word should be lowercase;
- Constant field names should consist of uppercase words separated by underscores;
- Local variable names follow similar conventions to member names, with abbreviations and short sequences of characters permitted.
Grammatical Naming Conventions:
- Instantiable classes are generally named with a singular noun or noun phrase. Non-instantiable utility classes are often named with a plural noun;
- Interfaces are named like classes or with an adjective ending in “able” or “ible”;
- Methods that perform actions are named with a verb or verb phrase. Methods returning boolean values usually begin with “is” or “has”;
- Methods returning non-boolean values or attributes of the object are usually named with a noun, noun phrase, or a verb phrase beginning with “get”;
- Field names typically follow the same conventions as class and interface names, using nouns or noun phrases.

Exceptions

Item 69: Use exceptions only for exceptional conditions

Exceptions should be used for exceptional circumstances, situations that are uncommon and unexpected. They should not be used for regular control flow.
Exceptions are not designed to be as fast as explicit tests, so using them for control flow can result in slower code execution.
Placing code inside a try-catch block inhibits certain optimizations that JVM implementations may perform.
The exception-based loop, which relies on throwing and catching exceptions for loop termination, is slower, obfuscates the code’s purpose, and can hide bugs in the code.
Exceptions used for control flow can mask bugs, making debugging more complicated. Bugs that would generate uncaught exceptions in a standard loop idiom may be caught and misinterpreted as normal loop termination in an exception-based loop.
An alternative to a state-testing method is to have the state-dependent method return an empty optional or a distinguished value (such as null) if it cannot perform the desired computation.

Item 70: Use checked exceptions for recoverable conditions and runtime exceptions for programming errors

Checked exceptions should be used for conditions from which the caller can reasonably be expected to recover. By throwing a checked exception, the API designer mandates the caller to handle the exception or propagate it.
Checked exceptions indicate recoverable conditions, and the API user should make an effort to handle them appropriately.
Unchecked throwables, which include runtime exceptions and errors, needn’t and generally shouldn’t be caught. They indicate situations where recovery is impossible, and continued execution would do more harm than good.
Runtime exceptions are typically used to indicate programming errors, such as precondition violations.
It may not always be clear whether a condition is recoverable or a programming error. In such cases, it is a matter of judgment for the API designer. If recovery is likely, use a checked exception; if not, use a runtime exception.
Checked exceptions should provide methods that aid in recovery from the exceptional condition.

Item 71: Avoid unnecessary use of checked exceptions

Overuse of checked exceptions can make APIs difficult to use.
The burden on the programmer increases when dealing with checked exceptions, as they must be handled in catch blocks or propagated outward, placing a burden on the API user.
Another approach is to refactor the method that throws the checked exception into two methods: one that returns a boolean indicating whether the exception would be thrown and another that performs the action. This allows the caller to check the state before invoking the action method.
Use checked exceptions sparingly, consider returning optionals, and throw unchecked exceptions if recovery is not possible or meaningful for the caller.

Item 72: Favor the use of standard exceptions

Reusing standard exceptions in your APIs and programs provides several benefits, including easier learning and usage, improved readability, and reduced memory footprint and class loading time.
The most commonly reused exception types are IllegalArgumentException and IllegalStateException. IllegalArgumentException is used when the caller passes an inappropriate argument value, while IllegalStateException is used when the invocation is illegal due to the state of the receiving object.
It is recommended to use specific standard exceptions like NullPointerException and IndexOutOfBoundsException for certain kinds of illegal arguments and states instead of using IllegalArgumentException.
ConcurrentModificationException is a reusable exception used to indicate concurrent modification of an object that was designed for single-threaded use or with external synchronization.
UnsupportedOperationException is another reusable exception used to indicate that an object does not support a particular operation, typically because it is an optional operation defined by an interface that the object implements.It is advised not to reuse the Exception, RuntimeException, Throwable, or Error classes directly. Treat them as abstract and avoid testing for them directly because they are superclasses of other exceptions.
When reusing exceptions, ensure that the conditions under which you would throw them align with their documented semantics. You can subclass a standard exception to add more detail if needed.
Choosing which exception to reuse can be challenging when multiple exceptions seem applicable. In such cases, follow the rule to throw IllegalStateException if no argument values would have worked, otherwise throw IllegalArgumentException.

Item 73: Throw exceptions appropriate to the abstraction

Higher layers should catch lower-level exceptions and throw exceptions that are meaningful and can be explained in terms of the higher-level abstraction. This is known as exception translation.
Exception translation is achieved by catching the lower-level exception and throwing a higher-level exception that encapsulates the lower-level exception and provides a more meaningful explanation of the problem.
Exception chaining is a special form of exception translation where the lower-level exception is passed to the higher-level exception as the cause. This allows for easy access to the lower-level exception and integration of its stack trace into the higher-level exception’s stack trace.
Exception translation should not be overused. Whenever possible, exceptions from lower layers should be prevented or handled within the higher layer itself.
If exceptions from lower layers cannot be prevented or handled, it may be appropriate for the higher layer to silently work around these exceptions, log them using an appropriate logging facility, and insulate the caller from lower-level problems.
Exception translation allows for better encapsulation, clearer APIs, and the ability to investigate the underlying cause of failures while insulating clients from lower-level exceptions.

Item 74: Document all exceptions thrown by each method

Documenting the exceptions thrown by a method is a critical part of the documentation required to use the method correctly.
Always declare checked exceptions individually and document the specific conditions under which each exception is thrown using the Javadoc @throws tag.
While the language does not require documenting unchecked exceptions, it is wise to document them carefully as they often represent programming errors. Familiarizing programmers with all the possible errors helps them avoid making these mistakes.
Do not use the throws keyword on unchecked exceptions.
It is ideal to document all unchecked exceptions that each method can throw, but in practice, it may not always be achievable due to dependencies on other classes that may throw additional unchecked exceptions.
If an exception is thrown by many methods in a class for the same reason, you can document the exception in the class’s documentation comment instead of documenting it individually for each method.
Failing to document the exceptions that your methods can throw can make it difficult or impossible for others to effectively use your classes and interfaces.

Item 75: Include failure-capture information in detail messages

The detail message of an exception should capture as much information as possible concerning the cause of the failure. It should contain the values of all parameters and fields that contributed to the exception.
Including pertinent data in the detail message aids in diagnosing the cause of the failure. It helps identify specific errors such as incorrect index values, boundary violations, or invariant failures.
It is crucial to avoid including security-sensitive information like passwords or encryption keys in exception detail messages as stack traces may be seen by multiple individuals during the debugging and issue-fixing process.
Do not write lengthy prose descriptions.
The detail message of an exception is primarily intended for programmers or site reliability engineers, not end users. Therefore, information content is more important than readability.
Requiring failure-capture information as parameters in the constructors of exceptions can ensure that the detail message automatically includes the necessary information. This centralizes the code for generating a high-quality detail message in the exception class itself.

Item 76: Strive for failure atomicity

Failure atomicity means that after an object throws an exception, it should still be in a well-defined, usable state, even if the failure occurred in the midst of performing an operation.
Immutable objects naturally achieve failure atomicity because their state is consistent when they are created and cannot be modified thereafter.
For methods that operate on mutable objects, one way to achieve failure atomicity is to check parameters for validity before performing the operation. This ensures that most exceptions are thrown before object modification commences.
Another approach to achieving failure atomicity is to order the computation so that any part that may fail takes place before any part that modifies the object. This ensures that failure occurs before any modification to the object.
A third approach is to perform the operation on a temporary copy of the object and replace the contents of the object with the temporary copy once the operation is complete. This is useful when the computation can be performed more efficiently on a temporary data structure.
Recovery code can be written to intercept a failure and roll back the object’s state to the point before the operation began. This approach is mainly used for durable data structures.
Failure atomicity is not always achievable, especially in cases of concurrent modification without proper synchronization. In such cases, objects may be left in an inconsistent state.
It is not always desirable or practical to achieve failure atomicity, as it can increase cost and complexity. However, it is often easy to achieve once the issue is understood.
API documentation should clearly indicate whether failure atomicity is achieved and what state the object will be left in after a method invocation.

Item 77: Don’t ignore exceptions

When an API declares a method to throw an exception, it is indicating that there may be exceptional conditions that need to be handled.
Ignoring exceptions by surrounding a method invocation with an empty catch block defeats the purpose of exceptions and can lead to disastrous results.
Ignoring exceptions is like ignoring a fire alarm and preventing others from addressing the problem. It is important to handle exceptional conditions appropriately.
There are situations where it may be appropriate to ignore an exception, such as when closing a file input stream where no recovery action is needed and the operation can proceed without issues.
If an exception is intentionally ignored, the catch block should contain a comment explaining the reason for ignoring it, and the variable used to catch the exception should be named ignored to indicate that it is intentionally disregarded.
Ignoring exceptions can lead to a program that continues silently despite errors, potentially causing failures at unexpected points in the code.
Letting exceptions propagate outward can at least result in a swift failure, preserving information for debugging purposes.

Concurrency

Item 78: Synchronize access to shared mutable data

Synchronization ensures that only one thread can execute a synchronized method or block at a time, preventing an object from being seen in an inconsistent state by other threads.
Synchronization not only guarantees mutual exclusion but also ensures that changes made by one thread are visible to other threads. It provides reliable communication between threads.
Reading and writing a variable (other than long or double) is atomic, but it does not guarantee that changes made by one thread will be immediately visible to other threads without synchronization.
Failing to synchronize access to shared mutable data can have dire consequences, even if the data is atomically readable and writable. It can lead to liveness failures and data corruption.
The java.util.concurrent.atomic package provides lock-free, thread-safe primitives, such as AtomicLong, for performing atomic operations on single variables, which can be used as a more efficient alternative to synchronized methods.
The best approach to avoid synchronization issues is to minimize sharing of mutable data. Either use immutable data or confine mutable data to a single thread. If sharing is necessary, ensure proper synchronization or use safe publication techniques.
Safe publication involves ensuring that an object reference is safely shared with other threads by using techniques such as storing it in a static field during class initialization or using volatile or final fields.

Item 79: Avoid excessive synchronization

Excessive synchronization can lead to reduced performance, deadlocks, or nondeterministic behavior.
Excessive synchronization should be avoided for performance reasons. Excessive locking can lead to contention and limit parallelism, as well as hinder the VM’s ability to optimize code execution.
Avoid unnecessary synchronization for mutable classes and document that they are not thread-safe if synchronization is omitted.
If a method modifies a static field and can be called from multiple threads, synchronize access to the field internally to prevent data corruption and ensure deterministic behavior.
Consider advanced synchronization techniques such as lock splitting, lock striping, and nonblocking concurrency control to achieve high concurrency when internal synchronization is necessary.

Item 80: Prefer executors, tasks, and streams to threads

The java.util.concurrent package introduced the Executor Framework, which provides a flexible interface-based task execution facility. It is recommended to use executors instead of writing your own work queues.
Creating an executor service is as simple as calling a static factory method. For example, Executors.newSingleThreadExecutor() creates an executor service with a single background thread.
Tasks can be submitted for execution using the execute() method of the executor service. The executor service takes care of managing the execution of tasks.
To gracefully terminate the executor service, the shutdown() method should be called. This ensures that the executor service will complete any pending tasks before exiting.
The executor service provides various additional functionalities, such as waiting for specific tasks or collections of tasks to complete, scheduling tasks to run at specific times, retrieving task results, and more.
Java provides static factory methods in the Executors class that offer different types of executor services, such as cached thread pools, fixed thread pools, and scheduled thread pools. These cover most use cases, but the ThreadPoolExecutor class allows fine-grained control over thread pool configuration.
Choosing the right executor service depends on the specific requirements of the application. For small programs or lightly loaded servers, Executors.newCachedThreadPool() is often sufficient. For heavily loaded production servers, Executors.newFixedThreadPool() or ThreadPoolExecutor with custom configuration is recommended.
The executor framework separates the unit of work (tasks) from the mechanism of executing it (executor service). This separation provides flexibility in selecting appropriate execution policies and allows easy changes if requirements evolve.
Working directly with threads is discouraged. Instead, think in terms of tasks and let the executor service execute them. This approach allows you to leverage the flexibility and control provided by the executor framework.
For a comprehensive understanding of the Executor Framework, it is recommended to refer to the book Java Concurrency in Practice by Brian Goetz.

Item 81: Prefer concurrency utilities to `wait` and `notify`

The higher-level concurrency utilities fall into three categories: the Executor Framework (covered in Item 80), concurrent collections, and synchronizers.
Concurrent collections are high-performance implementations of standard collection interfaces that internally manage their own synchronization. They provide high concurrency and state-dependent modify operations, making them ideal for concurrent applications.
Synchronizers are objects that enable threads to coordinate their activities. Examples include CountDownLatch, Semaphore, CyclicBarrier, and Phaser.
The concurrent utilities provide high-level abstractions that simplify common concurrency scenarios. For example, a ConcurrentHashMap can be used to implement a thread-safe canonicalizing map, and a BlockingQueue can be used as a work queue for producer-consumer scenarios.

Item 82: Document thread safety

Thread safety, or how a class behaves when its methods are used concurrently, is an important part of its contract with clients.
Failing to document the thread safety aspect of a class can lead to incorrect assumptions about synchronization, resulting in serious errors.
The private lock object should be declared final to prevent accidental un-synchronized access and is particularly useful for classes designed for inheritance.

Item 83: Use lazy initialization judiciously

Lazy initialization can be used to optimize the cost of initializing a class or creating an instance by deferring the initialization of a field until it is accessed.
Lazy initialization should only be used when necessary, as it can increase the cost of accessing the lazily initialized field.
Measure the performance impact of lazy initialization to determine whether it is beneficial for a specific scenario.
In the presence of multiple threads, lazy initialization requires synchronization to avoid bugs.
Normal initialization with the final modifier is usually preferable to lazy initialization for most fields.

Item 84: Don’t depend on the thread scheduler

The thread scheduler determines which threads get to run and for how long when many threads are runnable. However, the thread-scheduling policy can vary between operating systems, so relying on specific details of the policy can lead to non-portable programs.
To write a robust, responsive, and portable program, aim to keep the average number of runnable threads close to the number of processors. This minimizes variations in behavior caused by different thread-scheduling policies.
To keep the number of runnable threads low, ensure that each thread does some useful work and then waits for more. Thread pools should be sized appropriately, and tasks should be neither too short nor too long to avoid dispatch overhead.
Avoid busy-waiting, where threads repeatedly check a shared object waiting for its state to change. Busy-waiting puts unnecessary load on the processor and reduces the amount of useful work that can be accomplished. It also makes the program vulnerable to the thread scheduler’s behavior.

Serialization

Item 85: Prefer alternatives to Java serialization

Serialization in Java has proven to be risky, with problems related to correctness, performance, security, and maintenance.
The attack surface of serialization is significant and constantly growing. Deserializing objects can execute code from any type on the classpath, making it vulnerable to exploits.
Security researchers have discovered and exploited vulnerabilities in deserialization, leading to serious attacks, such as the SFMTA Muni ransomware attack.
Deserialization bombs can be used to mount denial-of-service attacks by causing deserialization of byte streams that take a long time to process.
The best defense against serialization exploits is to avoid deserializing untrusted data altogether. Serialization should be replaced with alternative mechanisms for translating objects to byte sequences.
Cross-platform structured-data representations like JSON and Protocol Buffers offer advantages such as portability, high performance, tooling support, and a large community.

Item 86: Implement `Serializable` with great caution

Implementing Serializable has long-term costs, as the serialized form of a class becomes part of its exported API. Changing the class’s internal representation while maintaining compatibility with the serialized form can be challenging and may limit the class’s evolution.
Serialization introduces the risk of bugs and security vulnerabilities. Deserialization acts as a “hidden constructor” and must guarantee invariants established by constructors, while also preventing unauthorized access to object internals.

Item 87: Consider using a custom serialized form

When the default serialized form is inappropriate for a class, it can result in several disadvantages, such as permanently tying the API to the current internal representation, consuming excessive space and time during serialization, and potentially causing stack overflows.
Regardless of the serialized form chosen, any synchronization imposed on the object’s methods should also apply to the writeObject method to prevent resource-ordering deadlocks.
Taking the time to design a custom serialized form that accurately represents the logical data and excludes unnecessary implementation details can lead to more flexible, efficient, and maintainable serialization. The default serialized form should only be accepted if it aligns well with the class’s logical state.

Item 88: Write `readObject` methods defensively

The readObject method is similar to a public constructor for deserializing objects, and it requires the same care as any other constructor to ensure the object’s validity and invariants.
To prevent invalid objects from being deserialized, the readObject method should perform validity checks after calling defaultReadObject.

Item 89: For instance control, prefer enum types to `readResolve`

When a class is serialized and deserialized, the readResolve method can be used to substitute another instance for the one created during deserialization.
The readResolve method can be implemented in a singleton class to return the original singleton instance and maintain the singleton property during deserialization.
The preferred approach for instance control is to use an enum type. Enum types guarantee that only the declared constants are the instances of the class, preventing any additional instances from being created during deserialization.

Item 90: Consider serialization proxies instead of serialized instances

The serialization proxy pattern involves designing a private static nested class, called the serialization proxy, that represents the logical state of the enclosing class in a concise manner.
The serialization proxy should have a single constructor that takes the enclosing class as a parameter and copies its data without performing any consistency checking or defensive copying.
Both the enclosing class and its serialization proxy should implement the Serializable interface.
The serialization proxy pattern eliminates the need for explicit field-level validation during deserialization and ensures that the invariants of the enclosing class are maintained.

Conclusion

While I did not find this book to be particularly entertaining, it is a very useful reference for Java developers. It covers a wide range of topics and provides a lot of useful information. I would recommend this book to anyone who is interested in learning more about the Java language and its best practices.

Resources and references

Effective Java, Third Edition - Joshua Bloch.
Java Concurrency in Practice - Brian Goetz.

Book summary: The Effective Engineer

Sébastien Lavoie — Sun, 16 Apr 2023 14:49:00 -0600

Introduction

This wonderful, mostly non-technical book is a compilation of the best practices and lessons learned by the author, Edmond Lau, during his impressive career as a software engineer. It is a must-read for anyone who wants to improve their skills and become more effective at what they do. The following are short excerpts from the condensed knowledge found in the book, sprinkled with some minor comments of my own along the way. These are extracted from the book’s takeaway sections and headings: I wholeheartedly recommend reading the work in its entirety for an immersive exposure to the fantastic storytelling capabilities of Edmond!

Book summary

Part 1: Adopt the Right Mindset

Focus on high-leverage activities

Leverage = Impact produced / Time invested

Leverage is the ROI for the effort put in.
Meta-skills are important to know where to focus one’s time and energy to translate more of one’s efforts into impact.
Leverage your time by always finding ways to perform activities quicker and making sure those activities will have a meaningful impact.

Optimize for learning

Make your learning rate a priority: it compounds like interest over time.
Study code for core abstractions written by the best engineers at your company.
Write more code.
Go through any technical, educational material available internally.
Master the programming languages that you use.
Send your code reviews to the harshest critics.
Enroll in classes on areas where you want to improve.
Participate in design discussions of projects you’re interested in.
Work on a diversity of projects.
Make sure you’re on a team with at least a few senior engineers whom you can learn from.
Jump fearlessly into code you don’t know.
Learn new programming languages and frameworks.
Invest in skills that are in high demand.
Read books.
Join a discussion group.
Attend talks, conferences, and meetups.
Build and maintain a strong network of relationships.
Follow bloggers who teach.
Write to teach.
Tinker on side projects.
Pursue what you love.

Prioritize regularly

Track to-dos in a single list.
Focus on what produces value directly.
Focus on the important and non-urgent.
Limit the amount of work in progress.
Use implementation intentions (if-then planning).
Prioritize routinely.
Write down and review to-dos.
Reduce context switches.

Part 2: Execute, Execute, Execute

Invest in iteration speed

Move fast to learn fast.
Invest in time-saving tools.
Shorten your debugging and validation loops.
Master your programming environment.
Get proficient with your favorite text editor or IDE.
Learn at least one productive, high-level programming language (scripting).
Get familiar with UNIX (or Windows) shell commands.
Prefer the keyboard over the mouse.
Automate your manual workflows.
Test out ideas on an interactive interpreter.
Make it fast and easy to run just the unit tests associated with your current changes.
Don’t ignore non-engineering bottlenecks.

Measure what you want to improve

Use metrics to drive progress.
Pick the right metric to incentivize the behavior you want.
Instrument everything to understand what is going on.
Internalize useful numbers (e.g., number of active users, requests per second, amount of data accessed and written daily, etc.).
Be skeptical about data integrity.
Log data liberally, in case it turns out to be useful later on.
Build tools to iterate on data accuracy sooner.
Write end-to-end integration tests to validate your entire analytics pipeline.
Examine collected data sooner.
Cross-validate data accuracy by computing the same metric in multiple ways.
When a number does look off, dig into it early.

Validate your ideas early and often

Find low-effort ways to validate your work.
- Approach a problem iteratively to reduce wasted effort.
Continuously validate product changes with A/B testing.
- Reduce the risk of large implementations by using small validations.
Beware the one-person team (you need feedback early to make sure you’re working on the right thing!).
Be open and receptive to feedback.
Commit code early and often.
Request code reviews from thorough critics.
Ask to bounce ideas off your teammates.
Design the interface or API of a new system first.
Send out a design document before devoting your energy to your code.
If possible, structure ongoing projects so that there is some shared context with your teammates.
Solicit buy-in for controversial features before investing too much time.
Build feedback loops for your decisions.

Improve your project estimation skills

Project estimation is one of the hardest skills that an effective engineer needs to learn. But it’s crucial to master.
To produce accurate estimates:
- Decompose the project into granular tasks.
- Estimate based on how long tasks will take, not on how long you or someone else wants them to take.
- Think of estimates as probability distributions, not best-case scenarios.
- Let the person doing the actual task make the estimate.
- Beware of anchoring bias (e.g., hearing about a low estimate from someone else may skew our own estimate later to be too low).
- Use multiple approaches to estimate the same task (e.g., estimate from past experiences, decompose a larger task into smaller ones and estimate each one).
- Beware the mythical man-month. Having more people on a project doesn’t necessarily mean it will be completed faster. Communication overhead, context switching, and other factors can slow down the team as much as $O(n^2)$ considering the number of people involved.
- Validate estimates against historical data.
- Use time-boxing to constrain tasks that can grow in scope.
- Allow others to challenge estimates.
Allow buffer room for the unknown in the schedule.
Define measurable milestones.
Do the riskiest tasks first.
Know the limits of overtime.

Part 3: Build Long-Term Value

Balance Quality with Pragmatism

Establish a culture of reviewing code.
- Catch bugs or design shortcomings early.
- Increase accountability for code changes.
- Provide an avenue for sharing best practices.
- Increase long-term agility.
Invest in good software abstractions to simplify difficult problems.
- Reduce the complexity of the original problem into easier-to-understand primitives.
- Reduce future application maintenance and make it easier to apply future improvements.
- DRY principle: solve the hard problems once and re-use the solutions multiple times.
Scale code quality with automated testing.
- Make large refactors with confidence.
- Offer executable documentation.
- Don’t test everything: focus on the most important parts of the code and make sure everything critical is covered.
Manage your technical debt.
- Pay it periodically. Incur it when necessary to meet deadlines.
- Repay the debts with the highest interest rates first (i.e., focus on what produces the most leverage).

Minimize Operational Burden

Embrace operational simplicity.
- Increased complexity introduces more potential single points of failure.
- New engineers face a steeper learning curve when learning and understanding the new systems.
- Effort towards improving abstractions, libraries, and tools gets diluted across the different systems.
- Do the simple thing first.
Build systems to fail fast.
- Pinpoint the source of errors.
- Make debugging easier by not masking your errors and by not deferring failures until later.
Relentlessly automate.
- Don’t underestimate the future frequency of the task.
- Internalize the time savings over a long time horizon.
- Automate mechanics over decision-making.
Make batch processes idempotent.
- At least, make processes “retryable” if they cannot be made idempotent.
Hone your ability to respond and recover quickly.
- Plan and practice failure modes.

Invest in Your Team’s Growth

Make hiring everyone’s responsibility.
- Identify which qualities you care about the most in your colleagues: keep the bar high.
- Make sure the recruiting process is effective.
- Adapt interview problems to different levels of difficulty based on the candidate’s experience.
Design a good onboarding process.
- Ramp up new engineers as quickly as possible (e.g., with codelabs).
  - Help the people around you be successful.
- Impart the team’s culture and values (e.g., have onboarding talks).
- Expose new engineers to the breadth of fundamentals needed to succeed (e.g., with mentorship).
- Socially integrate new engineers onto the team (e.g., give them starter tasks to feel they’re part of the team faster).
Share ownership of code (increase the bus factor).
- Avoid one-person teams.
- Review each other’s code and software designs.
- Make sure the team is exposed to different types of tasks and responsibilities regularly.
- Focus on readable and high-quality code.
- Share software decisions and architecture clearly.
- Make sure code and workflows are documented properly.
- Invest in teaching and mentoring.
  - The more effective the team becomes, the more freedom there is to work on new projects.
Build collective wisdom with post-mortems (e.g., at NASA, these become the user manual).
Build a great engineering culture.
- Optimize for iteration speed.
- Always automate.
- Use the right software abstractions.
- Nurture a respectful working environment.
- Allot experimentation time.
- Foster a culture of learning (CI/CD: continuous improvement, continuous delightfulness).
- Hire the best!

Conclusion

Edmond opened my eyes to the importance of being an effective engineer, showing what that entails and how to concretely achieve it. His extraordinary technical journey and knowledge of the field are evident throughout the book, and I found myself nodding in agreement with many of his points. He made me think about how I could apply his advice to my own work. Notably, the key concept of leverage resonated deeply with me and ended up being the cornerstone of the final project I built for my bachelor’s degree: ProductiviDo - An efficient task planner focusing on high-leverage activities.

Resources and references

Edmond Lau - The author’s personal website.
ProductiviDo - A simple and efficient todo-app built with React-Native.
The Effective Engineer: How to Leverage Your Efforts In Software Engineering to Make a Disproportionate and Meaningful Impact - goodreads.com.
The Effective Engineer - by Edmond Lau.

Reviewing the process of building a first React Native application

Sébastien Lavoie — Sat, 04 Mar 2023 12:41:00 -0600

Introduction

In the context of an assignment as part of a BSc in computer science, I had to come up with a React Native application. Specifically, I chose to build an app to help make and break habits through the use of a solid system that’s backed by science, which is presented through a large excerpt that accompanied the end result. It helps people to consistently stick to habits until they become an intrinsic part of their routine. Moreover, it helps to maintain an actual routine and to build a sense of accountability and capability in oneself, because big successes come from small steps taken in the right direction time and time again.

Instead of giving dopaminergic hits like other apps which strive to reward their users in one way or another for completing their tasks and goals or generating anxiety around items not being completed, this app focuses on giving full control to the user with a proven system to turn their habits into automated behaviours with ease. That was the idea, at least.

Concept development

The origin

The idea for this application came from listening to the Huberman Lab podcast from Dr. Andrew Huberman in an episode entitled The science of making & breaking habits. Starting from this excellent learning resource, further literature was devoured to inform the final concept, most notably Psychology of Habit from the Annual Review of Psychology and Intervention to Modify Habits: A Scoping Review published in Sage Journals.

Time restrictions

It was entirely feasible to come up with a simpler concept that would be doable in roughly 10 weeks. Like many popular applications on the market today, it is expected that new features will be added after the official release date once additional and valuable feedback is received from users. The core idea can and will be implemented in this short time frame. After all, this application can pivot towards a simpler presentation should there be a need to and as the Zen of Python says: “Simple is better than complex”. Niceties could be added to the user interface should time allow: however the initial development will focus on a narrower set of features so that the roadmap can be successfully followed and the final product delivered in a timely manner.

Main concern during development

A bad implementation of the system could mean that flexibility could be lost at the cost of ease of development. In other words, due to a strict, looming deadline, some shortcuts could be considered to cut corners in the underlying logic of the application to save time without affecting the quality of the product from the user’s perspective. Nonetheless, this could lead to more headache down the road when it becomes necessary to refactor the application in order to satisfy the need for a more modular approach, which would simplify further development.

Wireframing

Low-fidely as well as high-fidelity wireframes were created using Figma. An exhaustive presentation of the final application was included in an appendix, depicting the logo, the color palette, the typography used, a list of the UI elements and a complete and faithful display of all the screens. Technically, the high-fidelity wireframes were designed using Figma components and placed inside “frames” to export a PDF. Most UI elements were hand-drawn, except for a few free icons taken from the Material Design community. Numerous user flow diagrams were included to illustrate how the wireframing took shape once the logic to navigate between screens was determined.

User feedback

User feedback was critical to the success of this project. Users helped refine certain visual aspects, such as making sure there was enough padding between the different elements by testing with devices using a smaller screen.

Improvements to the original design

Other notable improvements included redesigning the user flow when creating a new programme so that it would be easier to follow. Instead of dealing with multiple habits at once, the slightly updated design made it possible to fully complete the cycle when adding a new habit. Similarly, once habits were added, the newer flow to tackle habit stacks made it simpler to navigate back to the main screen to confirm the selection of habits and to review the programme as a whole.

Without users inquiring about having a satellite view in the map, there wouldn’t have been any option to toggle the style of the map either. This became a needed addition in places where roads are sparse while the terrain remains easily recognizable.

Prototyping

Early iterations

While shaping the initial application alongside both wireframing and concept sketching, a few iterations of the different screens were designed from scratch. General ideas and concepts were discussed and analyzed to fit with the goal of creating a mindful habit tracker while taking into consideration hard requirements such as the time available for the development cycles.

The application started with a strong focus on accountability. However, it was quickly realized that the technical stack would need to be more ample and would necessarily require a higher budget to fund the use of online services to store a database of “habit contracts” that users would create and optionally make public on their own. Needless to say, the single developer involved in the creation of this application did not have the resources nor the willingness to deal with legal issues that would arise from managing such personal information.

Privacy concerns

Additionally, there would not have been the easy possibility of making accessible all the data from a habit contract since personally identifiable information such as location, accountability partner names and so on would be revealed to the world. For these reasons and many more, the set of features relating to accountability was removed from the application to ensure that user privacy would be dealt with properly at this scale of operation.

In this regard, to avoid any potential issue with privacy, no data whatsoever is stored remotely. React Native Async Storage was used to store data locally on users’ devices, even removing the need for an internet connection (except for the need to find the user location and set an address, which is an optional step when creating a programme).

To make use of the async storage, Redux Toolkit was used to deal with all the state management and is responsible for dispatching actions throughout the application. Importantly, users do not need to leave the application open at all times since Redux Persist was used to store Redux data to the async storage and to rehydrate it when the application is re-opened.

Development

As we can now attest, a lot happened before developing the actual application, but this fact did not make the development itself less interesting to narrate! In a little over two months, this React Native application started from nothing and grew to a fully-fledged product.

Initial setup

The initial steps taken were all about setting up the project correctly with most of its JavaScript dependencies, beginning with a .gitignore file to prevent adding secrets and other unneeded artifacts to the project, an app.json to define the metadata required to publish an application with Expo and a package.json listing all the major dependencies to be installed with Yarn in this case. Secondly, Reactotron was set up with Redux to provide access to the application state in real time. Strict mode was then enabled to help avoid bad programming practices and third-party libraries were silenced with React Native’s LogBox module.

State management

Once the basic setup was in place, Redux Persist was added to work with React Native Async Storage. Then, the testing setup was integrated into the project before adding any screen to the application using Jest configured with mockAsyncStorage to avoid issues with the persistent storage while testing. Despite barebone Redux being useful on its own, Redux Toolkit provides some useful utilities and shortcuts that were taken advantage of, including createSlice to easily “slice” parts of the state to retrieve and store information in isolation, combineReducers to list all the slices of state in the same “root reducer”, and configureStore to connect the persistent storage to the Redux store and set up access to the store itself in the rest of the application.

Styling the app

Before adding logic to the project, react-native-extended-stylesheet was used to add functionality to the way the styling could be done by having the ability to reference variables, do calculations inside CSS fragments, etc. Styles were set globally right from the start to avoid code duplication and where deemed useful, some screens had specific stylings applied exclusively to them from separate files.

As a next step, the general structure of the app was set up with a status bar and a SafeAreaView component to make sure that rendering happened within safe limits on the screen. In order to style the application and render different screens, React Navigation was used to make the navigation possible and to initially render a fixed screen at a time while it was being designed based off the high-fidelity wireframes. One by one, the screens were designed with all the buttons in the right place and some dummy content to render all the different UI elements. This led to using a multitude of third-party libraries, including notably react-native-maps to work with native maps on both Android and iOS devices.

Premature refactoring

With the screens in place, the code was re-arranged to allow for better flexibility while developing. New directories for components were created, and the smallest components of all were extracted from the larger ones to start splitting concerns and have specific destinations to receive parts of the states via Redux selectors.

Global styles were then cleaned up considerably and local styles were mostly eliminated as they were being re-used elsewhere without notice. At this point, without yet having anything useful to demonstrate, users reviewed where the application was going and suggested possible improvements while it was still easy to move UI elements around without breaking anything.

Once all the screens were designed and ready to include programmatic logic, the screens were then linked together so that the user could –without doing anything useful yet– navigate from one page to another. After that, a menu was included in the top-left corner to navigate to specific pages from anywhere within the app. This behaviour was later customized so that some pages would show the “back button” (a chevron icon pointing to the left) to make going back to the previous screen easier than scrolling to the bottom of a longer page. On top of this, some screens received a special treatment where the menu is hidden entirely, such as when deleting a programme from the app. While not strictly required, this made it obvious to the user that they should focus on the content of that particular screen.

Customizing UI elements

At this stage, the application was ready to receive its final touch-ups to look as polished as the expected product. This included getting our hands on React Native Bouncy Checkbox to give a more joyful feel to the most boring parts of the application and using React Native DateTimePicker to input the time of the day in a more meaningful and straightforward way.

Setting up simple states

Using the useState hook from React and useDispatch from Redux, basic states were set up to toggle the styles of buttons and to allow basic interactivity, such as expanding and collapsing sections of the user interface. Using some dummy content in the Redux store, other simple events were taken care of, for example removing a habit, creating a new habit and so on. With the basic logic and states in place, more complex flows were added, such as the need to loop through habits to display them as “habit stacks” or looping through the days in a programme to display three phases for each day, each phase containing possibly multiple stacks of habits and each habit being a self-contained source of data with a table displayed with react-native-tableview-simple.

Testing on different platforms

Although admittedly a bit late in the process, the application was then tested fully on Android and in the Expo website. Trading off speed of development for the inconvenience of guaranteeing a pixel-perfect application at each step was most definitely worth it since very few changes were required in the end, this being the main idea behind using something like React Native in the first place so that code can be written once and for all. Despite all these promises, some issues related to dependencies were faced with big, red messages showing an alarmingly large list of errors in the Expo Snack interface. Thankfully, it was only a matter of upgrading locally to match the latest SDK available at the time from Expo (v46.0.0).

There were some minor issues observed on different platforms which were fixed on the development machine using simulators. For instance, the map view was somewhat laggy on iOS originally, so instead of moving the marker around on the map, it was decided to leave it fix on top of the center of the map with a greater zIndex to make it visible at all times in order to have only the map view to re-render each time the map was moved. On Android, the time picker was a bit fiddly to work with and so to circumvent the problematic situation, the picker would be shown once automatically to the user when the related screen was opened, removing the need to display a space for the picker on screen and instead having it fill out the screen in the style of a modal.

Improving the developer experience with Redux

Reactotron was useful in the beginning but started to crash very often at this point of the development. After un-starring the project on GitHub and starring React Native Debugger instead, things were smooth again. Reactotron required messing a little bit with the testing setup by removing everything related to it first, then React Native Debugger made it all simple again thanks to its seamless integration with Redux which does not need to be injected as middleware when configuring the Redux store.

Refactoring again

Now that most of the logic was implemented, smaller components were created from the monstrous ones created thus far. In most instances, components were already existing on their own but they were all positioned in the same file without being prefixed with the keyword export. Equally important, all ESLint warnings were cleared up one after the other to avoid re-renders and unused bits of code. Because Expo tends to be picky with missing import statements, import React from ‘react' was carefully added to almost every file in the project where it was needed.

Last-minute cleanup

Finally, once the application was usable and the source code a little more pretty, import statements were grouped by categories, warnings from third-party libraries were silenced with LogBox, spacing around paragraphs was reviewed and optimized and the app was scrutinized once again to look for the presence of bugs.

Unit testing

Navigating

While the development took place, testing happened all along. The first couple of tests to be included in the test suite were added while validating that navigation occurred correctly between screens. At once, tests were also added when creating the screens to make sure that they would render consistently in the UI by creating snapshots with Jest.

Utility functions

More concrete unit testing happened with the utility functions being called by the actions from the Redux store. Because these functions define what happens when an action is dispatched with Redux, there is a lot of potential for bugs to creep in. To make debugging less painful in these cases, dummy and minimal states were set up as part of the unit tests to check for inconsistencies against the input value and the expected one. While Jest offers a complete range of features, the matchers used most often from this library were toBe to check for exact values (e.g., Boolean and numbers) and toEqual to assert a deep equality on objects and arrays. Mock functions were set up with Reactotron, but that need went away when the developer migrated to a more stable piece of software (i.e, React Native Debugger).

Database & asynchronous code

In the future, should this application be deployed to the cloud, there will be a need to use setup and teardown functionality against a database and to add tests for asynchronous calls. This was not required for the present version of the application, which focuses on allowing users to be as independent as possible from the Internet while respecting their privacy.

Evaluation

Overview

All in all, despite a strict deadline and few (more truthfully, close to none) monetary resources, it was possible to deliver a working solution to a tangible problem, namely that of increasing productivity by knowing what is most valued in one’s life circumstances. While the application can be considered feature-complete, it could be the beginning of a long journey towards sharing an application that has real potential to improve people’s lives by solving a problem that is hard to pinpoint without enough awareness.

Technical aspects

On the technical side of things, it would have been preferable to split the state in a more logical way right from the beginning. Even though a general idea of the state was drawn inside an IDE by creating a JSON object to represent the schema to be adhered to, the state was allowed to mutate along the way, which caused issues by having to move reducers around and by having to update the selectors to look at a different hierarchy of the state. By contemplating the real meaning of a Redux “slice” in which the state is meant to be entirely independent from other slices of the state, it makes it obvious that some types of object will remain independent while others will eventually require to communicate with their siblings. Thinking this logic through before starting to make things happen in the UI is, in hindsight, a fruitful endeavour.

Communicational aspects

On the soft skills side, a mistake was made by not sharing the original concept with users sooner. Although the final product would likely resemble what it ended up being, interactivity in the application could have been designed better regarding some aspects. For example, the component responsible for displaying the frequency of execution could very well become an actual calendar. Designed in a compact way, a calendar surely would be more familiar to most users instead of having a list that needs to be scrolled into view. Functionally speaking, ticking a box in a calendar-like view would be exactly equivalent to checking off a box next to a row item in a list, but the presentation would be more familiar to most users and, quite understandably, more intuitive too.

Demo of the app

Resources and references

If you like this type and format of content, you might also enjoy a follow-up story about building a task planner, also brought to life with the help of React Native!

Building the next Facebook in a couple of weeks (as a humble experiment)

Sébastien Lavoie — Fri, 14 Oct 2022 18:49:00 -0500

Introduction

The following is a report that was part of a project I delivered during my BSc in computer science, reformatted from LaTeX to Markdown. While not being utterly interesting (unfortunately, the source code cannot be freely published due to the nature of the work…), it gives a glimpse of my thought process while building this Django website. I have learnt a few lessons from it and maybe, with some imagination, you might too :). To read a nicely formatted version of this article, you can refer to the original report instead.

Meeting the requirements

In order to meet all the requirements when building this application, a few different techniques, strategies and concepts were put to good use. Firstly, all Django views are class-based and rely on Django REST Framework to reduce the boilerplate needed to write the views. One such (simple) view is shown below:

 python
 
 
 
class Profiles(APIView):
 """Used when listing all profiles at once."""

 # allow everyone to see all basic profiles
 permission_classes = [IsOwnerOrReadOnly]

 def get(self, request):
 profiles = UserProfile.objects.all().order_by("-id")
 serializer = ProfileSerializer(profiles, many=True)
 return Response(serializer.data)

All views in turn rely on serializers to parse incoming data and to transform it into appropriate structures that can be used internally by Django. Some serializers are obviously more complex than others, but for demonstration purposes and for the sake of brevity, an example of a class-based serializer is reproduced here:

 python
 
 
 
class UserPublicSerializer(ModelSerializer):
 """Serializer the User model to get back user information that can
 safely be shared around in the application to other users."""

 # We need easy access to the profile image when listing all profiles
 # at once
 profile_image = SerializerMethodField()

 class Meta:
 model = User
 fields = ["id", "name", "profile_image"]

 def get_profile_image(self, obj):
 return obj.userprofile.image.url

The actual implementation details for signals, forms, URLs, other views and components can be explored in the source code submission provided with this report.

When it comes to version control, Git was used religiously with GitHub to host a private repository. The Conventional Commits specification was followed to help create meaningful commit messages.

Defensive coding strategies were used, both in Python and in JavaScript. In the backend, exceptions were raised directly where no appropriate actions could be taken:

 python
 
 
 
def clean_email(self):
 """Verify that the email is available."""
 email = self.cleaned_data.get("email")
 query_set = User.objects.filter(email=email)
 if query_set.exists():
 raise forms.ValidationError("email is taken")
 return email

In the views, conditional statements were sprinkled everywhere where necessary to avoid expansive computations (e.g., processing a profile image when the request is invalid) and to return early from functions:

 python
 
 
 
# ...
if profile is None:
 return Response(
 "Profile not found!", status=status.HTTP_404_NOT_FOUND
 )

if profile.user.id != request.user.id:
 return Response(
 "You do not have the required permissions "
 "to update this profile!",
 status=status.HTTP_401_UNAUTHORIZED,
 )

serializer = ProfileUpdateSerializer(profile, many=False, data=data)
if serializer.is_valid():
 serializer.save()
 return Response(serializer.data)
# ...

On the frontend, a kindred approach was used to assemble robust requests:

 javascript
 
 
 
try {
 const response = await axios.post(
 `${process.env.REACT_APP_API_URL}/auth/jwt/verify/`,
 body,
 config,
 )
 if (response.data.code !== "token_not_valid") {
 dispatch({
 type: AUTHENTICATED_SUCCESS,
 })
 } else {
 dispatch({
 type: AUTHENTICATED_FAIL,
 })
 }
} catch (err) {
 dispatch({
 type: AUTHENTICATED_FAIL,
 })
}

With security in mind, views were by default protected and accessible only to authenticated users unless a statement such as the following allows all users to read an endpoint:

 python
 
permission_classes = [IsOwnerOrReadOnly]

Requirements were also exceeded by using advanced techniques and tools: Docker, Docker Compose, Poetry, React, Redux, Material UI as well as setting up an SMTP backend to send emails to users were some of the additions made to the project to prove this claim – these will be covered in more details in later sections of this report.

While being a simple application considering time constraints and budget, the solution brings some novelty to the table, making it a valid application that is production-ready while also serving as an excellent prototype for a much bigger project. Some interesting features not commonly found elsewhere include:

Being able to chat in real-time with all users of the platform in a time-sensitive context in a way that supports high concurrency with Django Channels and Redis. In other words, messages disappear automatically after a specified delay, making for a special chat room where one lives in the moment.
The application allows users to easily follow others to then be able to quickly search through their posts by filtering them. The added tweak resides in the fact that all posts are given “total points” without “up” or “down” counts being disclosed directly.
Posts can be hidden at any time by users yet remain viewable by showing the author as being anonymous.
The colourful profile list view allows filtering of all the profiles at once by name and shows distinctive information about users, such as the list of people they follow or the list of people who are following them as a non-obstrusive modal (Material UI pop-up).

As will be discussed in the section State of the art in web development, this solution follows best practices and aims to represent the state of the art, both in terms of technical execution and in the contemporary look of the final product, which is brought into the spotlight in the following figures.

Running the application

Installing and running the web application

Accessing user accounts

All accounts are registered with the same password: pass1234.

To log in, simply use one of the following email addresses:

 
 
 
admin admin@gmail.com (superuser for Django admin)
Andrea andrea@gmail.com
Bob uol.grader@gmail.com
Hider hider@gmail.com (hidden profile)
Mia mamamia@gmail.com
Suliman suliman@gmail.com
Tuan tuan@gmail.com

If you set the SEND_ACTIVATION_EMAIL setting to True in the configuration file socialnetwork/settings.py, emails will be sent from the account uol.grader@gmail.com, which is a real account created to manage emails for this application.

Running locally

Backend

To avoid a layer of abstraction as setting up Redis on some systems may be more difficult, Django’s backend is used for the channels by default. Should you require to use Redis instead, please follow the instructions in settings.py:

 python
 
 
 
CHANNEL_LAYERS = {"default": {
 "BACKEND": "channels.layers.InMemoryChannelLayer"}
}
# To use Redis locally, please comment the above line and uncomment the
# following two dicts:
# CHANNEL_LAYERS = {
# "default": {
# "BACKEND": "channels_redis.core.RedisChannelLayer",
# "CONFIG": {"hosts": [("redis", 6379)]},
# }
# }

# CACHES = {
# "default": {
# "BACKEND": "django_redis.cache.RedisCache",
# "LOCATION": [("redis", 6379)],
# "OPTIONS": {
# "CLIENT_CLASS": "django_redis.client.DefaultClient"
# },
# }
# }

Then, a simple way to get Redis running would be to launch it with Docker:

bash

docker run -d -p 6379:6379 redis

Moving on to installing Django and the rest of the dependencies for the backend:

 bash
 
cd backend

# Activate a virtual environment, e.g.:
python3 -m venv .venv && source .venv/bin/activate

# Install dependencies
pip install -r requirements.txt

# Run the server on port 8000 with existing data
python manage.py runserver

Dependencies can also be installed with Poetry thanks to the provided files pyproject.toml and poetry.lock with the following command:

bash

poetry install

To keep the Docker image simple, pip is used to install the requirements, although beware that Poetry’s dependency resolver will be more predictable in a production application.

Frontend

 bash
 
cd frontend

# Install dependencies
npm install

# Run the server on port 3000
npm run start

Access the application at http://localhost:3000.

Using Docker

A docker-compose.yaml has been prepared to launch all the required parts at once (Redis, Django and React).

bash

docker-compose build
docker-compose up -d

Some other useful commands:

 bash
 
docker-compose down
docker-compose logs
docker-compose ps -a

This container could be set up to run in a cloud infrastructure such as Google Cloud Platform inside a virtual machine. This setup goes above and beyond what was taught in this module, though, therefore resources were not allocated to the actual deployment.

Local deployment

An alternative to the (costly and unfunded) deployment on GCP can be done on a local machine, using Docker to containerize the application and a service like Cloudflare Tunnel to allow traffic from the external world to reach the application, effectively converting the machine into a server.

Summary of the steps needed to deploy locally

Buy/use a domain name and configure its DNS with Cloudflare.
Update Django’s settings.py to allow the domain name to reach the Django application – avoiding CORS issue when localhost:3000 tries to connect to localhost:8000 using that domain – (i.e. ALLOWED_HOSTS).
Install the CLI tool cloudflared, log in to your Cloudflare account with cloudflared tunnel login.
Create a new tunnel: cloudflared tunnel create test-tunnel.
Route that tunnel: cloudflared tunnel route dns <T> <N>, where <N> is the web address to use as a sub-domain and <T> is the tunnel ID.
Add this sub-domain to the DNS management console in the Cloudflare dashboard.
Configure Cloudflare to use the web application:

 
# ~/.cloudflared/config.yml
url: http://localhost:3000
tunnel: SOME_ID_GIVEN_BY_CLOUDFLARE
credentials-file: ~/.cloudflared/SOME_ID_GIVEN_BY_CLOUDFLARE.json

Finally, run the tunnel with cloudflared tunnel run and go to the configured website at its subdomain (e.g., name.domain.com).

Functionality implemented

This application implements all the necessary functionality at the account management level with Djoser. In effect, users can:

Sign up for a new account;
Activate their account via email upon creation (this behaviour is disabled by default for convenience but can be reactivated as mentioned earlier);
Reset their password via email;
Log into their account;
Change their displayed name;
Make their profile hidden to other users;
Log out from their account;
Delete their account permanently by validating their password.

User profiles provide the following features:

Number of followers as well as a complete list of followers shown;
Number of people that a user is following as well as a complete list of these people;
Number of points a user has “given” to posts they like;
Number of points a user has “taken” from posts they dislike;
Where the user is located;
Each user can display a biography;
The date at which a user joined the social network;
The latest date at which the user updated their profile;
A “status” field to prominently feature a statement, which is also shown in the profile listing;
A list of posts written by the user, where the owner can create new posts, delete, edit or hide existing posts so they are no longer displayed publicly in their profile nor in the list of public posts;
An image gallery the user has uploaded, where the owner can add or remove pictures;
The ability for each user to delete their account, effectively cascading a delete of their interactions in the application such as their posts, profile image, etc.;
Users can edit the information they share with others;
To avoid profile pictures slowing down navigation – most notably in the profile list view – all images are automatically resized to maximum 300 by 300 pixels when a user uploads a new image;
Finally, users can also hide their profile entirely – doing so will then mark all their posts as being anonymous and they will not be reachable by any other user of the network.

Following the “profile” view, the “profile list” view (linked with the text “Meet” in the navigation bar) shows all public profiles with most of the information users share. To view the details of a profile (such as a birth date and the date of the last edit of a user), it is necessary to be authenticated, otherwise unauthenticated users will be redirected back to the “profile list” view and an error alert will be flashed. Additionally, the number of followers and people users are following are displayed right from the list view and allow to click to retrieve a complete list of people who are followed. The different icons shown on this page really pop out: the more followers a user has, the bigger the number of people shown in the actual icon representation; also, an icon indicating a value of “0” is slightly greyed out for readability and to let users know they won’t be able to interact with these icons.

As a distinctive mark of interest, everything belonging to a user is marked with a fingerprint icon: when a user views their own profile or posts, they will be able to quickly identify that this information is theirs as long as they are logged in.

Posts implement a basic editing view and allow their author to hide them. Furthermore, posts receive and lose points based on the opinions of users in the network and will end up with positive or negative scores, similar to what Reddit and Stack Overflow have done but without splitting the count in either direction.

The “Read” page – to be found in the navigation bar – shows posts from people users follow at the top, allowing to filter the content of posts based on their interests. The total number of points is shown next to each post entry. As a section section on that page, users have the ability to see a list of public posts as well as search for keywords of interest, just like in the first section just described.

On any page that gets longer than 500 pixels when scrolling down, there will be a little arrow icon allowing users to swiftly glide to the top of the page.

When clicking on the avatar icon displayed in the top-right corner of the application, a “drawer” (in Material UI’s terms) will open to allow users to quickly jump to their profile and to log out of the application.

Behind the scenes, administrators and superusers can be created with full privileges to modify and delete other accounts. These special accounts will be hidden by default from the public-facing frontend, although their profile can later be made public and used as any other account on the website.

Last but not least, there is a public chat room where all users of the application can go to have a spontaneous discussion. In fact, the web application is programmed to automatically discard messages that are older than 20 seconds (by default), allowing users to share special moments that are very much unique and time-limited while avoiding being overwhelmed by content. In the event that a constant flow of messages might be entering the chat room at a given time, users will still have the ability to scroll up in the view (assuming they read fast enough). If they do not scroll up, the view will always focus on the latest incoming message so their attention span will be maintained on this very moment. Every time a user reloads the page or goes somewhere else in the application, the content of the chat window will disappear on purpose, because good discussions are meant to happen in real-time on this website. Users will not be distracted by anything else other than a little avatar icon and people’s names so they can know at a glance who is interacting with them… but they will need to have a good memory to remember everything. After all, the idea behind this chat application really is to exchange some good thoughts in the moment. What lives on is only the memory of time well spent with others.

At the API level, users can interact with the application directly via API endpoints, provided they send their authentication token should a view have a requirement in this regard. These endpoints are listed succinctly below, but a file openapi-schema.yml has been provided in the backend directory for a complete reference.

 
 
 
accounts/ profiles/
accounts/ profiles/<int:user_id>
accounts/ profiles/<int:user_id>/image
accounts/ profiles/<int:pk>/followers
accounts/ profiles/<int:pk>/followers/add
accounts/ profiles/<int:pk>/followers/remove
accounts/ profiles/gallery/add
accounts/ profiles/gallery/remove/<int:gallery_pk>
accounts/ ^media/(?P<path>.*)$
api/ posts/<int:pk>/
api/ posts/author/<int:pk>/
api/ posts/<int:pk>/up/
api/ posts/<int:pk>/down/
api/ posts/<int:pk>/pointsby/
api/ posts/

Executing the unit tests

To run the unit tests, it is a matter of activating a virtual environment (as described in the section Running locally) and using the built-in test command provided by Django in the following manner, which must be run from within the backend directory:

 bash
 
 
 
$ python manage.py test
Found 119 test(s).
Creating test database for alias 'default'...
System check identified no issues (0 silenced).
..............................................................
--------------------------------------------------------------
Ran 119 tests in 0.584s

OK
Destroying test database for alias 'default'...

Logical approach

Frontend framework: React

The application was designed with the goal in mind of having a modern frontend framework relying on an excellent CSS framework to provide icons and other components. Speaking of components, React was chosen for its strong focus on designing small components that can be reused. Taking advantage of this approach, some components such as the “profile intro” (the box showing the main information regarding a user profile) are literally displayed on different pages, such as when viewing a single profile or when viewing a list of profiles.

Similarly, a “post” entry represents the exact same React component when viewed in isolation or when browsing for many more posts. When search bars are used, these share the same characteristics and behaviors across the page while managing a different state.

React components and icons: Material UI

On the design side, the presentation of the website was made much more consistent with the help of Material UI, which provides many ready-made React components. The whole application was built around the concept of adding building blocks from Material UI such as a “box”, “button” or “typography” to guarantee that all aspects of the application look homogeneous and with the same modern style.

Components such as a “drawer” and a “navbar” really sped up development and were expected to be used extensively right from the start.

API: Django Rest Framework

Since the use of Django was mandatory for the backend part of this work, Django REST framework was an ideal choice considering the need for an API to be directly accessible to end-users and to benefit from an easy-to-use toolkit to build an API that can be accessed from React without breaking a sweat.

Django REST Framework simplifies data serialization and even allows dealing with nested serialization in a straightforward way. Because it comes with powerful features for authentication and permission management, it was also a great option for this specific use case which depend easily on these sets of features which would arguably be very complex, error-prone and time consuming to implement manually.

User management: Djoser

Beyond Django’s default user management system, there was a desire to extend the default behaviour, for instance to use an email address instead of a username to authenticate users. Djoser makes these changes and much more a lot more simple, adding easy management of logins, logouts, password confirmation resets, activation emails and so forth.

By leveraging Django’s built-in features with Djoser, the user model can be extended with ease and flexibility.

Asynchronous communication

To build the chat application, Channels and WebSocket were considered as one of the best combination of libraries because they are simple to use and allow high concurrency to be possible. Channels makes it easy to create an asynchronous WebSocket consumer to deal with many users connecting to possibly many different groups: it is therefore an ideal option for a growing social network.

The WebSocket library in JavaScript is then used to communicate with Channels via the WebSockets technology, sending and receiving messages with additional payload to extend the functionality of the chat application.

Deployment: Docker, Cloudflare Tunnel

To deploy an application to production, Docker Compose is a robust and reliable solution, especially for this use case where different microservices can be pieced together seamlessly, sharing resources across a single network on different ports.

Docker makes it possible to containerize parts of the application (i.e., the backend, frontend and in-memory cache database) to be run potentially on many virtual machines in the cloud and Docker Compose makes the process of orchestring an application such as this one much more accessible than configure web servers from start to finish.

Poetry, a package and dependencies management system for Python, allows requirements to be pinned to specific versions and ensures reproducibility is achieved, which is critically important when it comes to deploying to production. Its dependencies resolver is much more reliable when the exactitude of a list of requirements must be met.

In terms of deploying a real application on a small scale, Cloudflare Tunnel was also considered as it is useful to share prototypes during development and to instantly deploy a local server. Furthermore, it can work hand in hand with Docker Compose and Poetry since a container can be run in the background and the application can be exposed on multiple ports locally without having to configure firewalls.

Because pip is traditionally used in the Python world to manage requirements, the command pip freeze was used to generate a standard list of pinned dependencies, although for an actual deployment to production, an open source alternative like Poetry would likely be preferable.

Formatting: Prettier, Black, Vulture, Pylint

While code formatting is of utmost importance for readability, it is also a tremendous waste of time to try to do it manually. For this reason, multiple tools were used to aid with this gargantuan task. Prettier was used on the frontend side to format everything related to JavaScript and CSS.

On the backend side, Black was the primary formatter used. Its use was complemented with Pylint, which helps to clean up and refactor the code by finding hard-to-spot warnings, such as unnecessary return statements or the use of more complex expressions when an equivalent such as a list comprehension could be used.

Finally, to keep the code tidy and alive, Vulture was used to find “dead” code not in used anywhere else. Even if code is well formatted, code that is not needed become a liability. In the source code world, less really is more.

API calls: Axios

The Axios JavaScript library was used for the attractiveness and flexibility of its API. Because it supports fully the Promise API and allow making asynchronous calls with the async/await syntax, it makes for a good tool to communicate with the backend. It has extensive documentation and excellent support on various web platforms including GitHub, which is an important consideration when thinking about deploying an application in the real world.

There are new and shiny technological options all the time, but a battle-tested software like Axios remain a competent tool apt for the job.

State management: Redux and Immer

Despite the Context API from React being powerful and easy to use, Redux and Immer really make dealing with state management a breeze. Redux on its own is one of the best in its category to provide state management at scale: its documentation is also fantastic and detailed and its popularity on Stack Overflow is only a plus in its favour should there be a need to fix issues while using it (because there will inevitably be such a need). The Redux workflow with actions being dispatched with types makes sense and adding new features is very fast without having to change existing pieces of states when combining reducers.

One possible drawback of using plain Redux to manage the store is that states can be mutated and unexpected consequences may ensue. For this reason, Immer was included so that dealing with a Redux store is stripped down to a direct object assignment, where Immer takes care of updating items and even nested items as needed.

Design and implementation decisions

Because deployment with Docker was a primordial concern, the application was built with a backend that is completely decoupled from a frontend. In fact, with just a little bit of ingenuity, Django could be swapped entirely with another web framework as long as the same endpoints are provided to the frontend with the necessary technologies (such as web sockets) being implemented. In a similar way, the frontend, which is built primarily with React, could be upgraded to another popular alternative such as Vue.js. This gives flexibility and should there be a greater need for optimization once the social network takes off for a real spin, there is the possibility to bundle React with Django by building minimized versions of the assets and shipping everything as a Docker container that can be run on powerful machines in the cloud.

Beyond this original concern, more concrete decisions were taken. For instance, Django signals were used to simplify how user profiles are created. Instead of allowing users to create a profile from their user account, the experience was unified by transparently automating this process whenever a user account is created such that it is attached to a newly created profile associated with it. Leaving the option to the end-user of manually creating a profile could be confusing at best and lead to unexpected results at worst, where user accounts and user profiles IDs become out of sync, which is a nice side-effect of having them created as well as deleted at the same time.

From the start, Djoser was chosen to override specific built-in functions performed by Django and to facilitate email management. To integrate Djoser with the administrative bits of the application – namely, the Django admin interface – Django forms were used to customize how users are to be registered as well as superusers and staff accounts. By overriding the user creation, it is now possible to update parts of the process with simple configuration variables implemented by Djoser, such as forcing users to retype their passwords or not.

With Django models, the base model became a UserAccount, which allow users to join the network. However, the functionality related to account management is kept separate from other features of the application by interfacing with the accounts using a UserProfile model on which a user field exist. In turn, this model integrates other parts of the models, such as many-to-many relationships with followers and a gallery image model, which is also linked as a many-to-many relationship. Another part of the application implements a Post model where the author field corresponds to a UserAccount, effectively linking a different part of the application to the user accounts. Again, a Post then interfaces with other models to take points into account, which are also linked to user profiles.

To manage all kinds of POST requests as well as to guarantee that models communicate as expected between each other, serializers using Django REST framework features were used in the process. These generally use a SerializerMethodField to define more complex behaviors, such as getting all non-hidden followers for a single profile.

Views were implemented with DRF APIView, bridging the gap between a RESTful API and the ability to define simple views using methods like get and delete. Views rely on serializers and specify a permission_classes list of custom values, which were implemented separately in a permissions.py file. This allows customizing the default behavior so that by default all views require authentication unless explicitly stated otherwise by these permission classes. This is also more secure, preventing users from inadvertently being able to access what should have been protected resources.

In terms of API definition, simplicity was key: every endpoint regarding authentication is accessed under /auth while our specific implementation of user accounts is accessed under /accounts. Additionally, all endpoints to deal with user posts are to be found with an endpoint starting with /posts. Finally, there is another set of endpoints to work with profiles, these being accessible under /profiles.

The Django chat application, due to its asynchronous nature, uses different technologies behind the scenes, including notably WebSockets. Channels was chosen to implement this portion because it extends Django’s capabilities beyond HTTP and is ideally suited for chat applications like the one being built as part of this social network.

Evaluation of the final product

What worked well

Opting for React made for a flexible framework that behaves extremely well with Django. React has complete documentation and extensive support offered on different medium, making it ideal for a robust and complex web application. Coupled with Material UI, the frontend aspect of the work was taken care of and didn’t need particular attention when developing. For any basic functionality that was not available out of the box in JavaScript, the Lodash library was used to make object comparisons and such much more readable and understandable when multiple conditions were being evaluated in a JSX expression, often with ternary operators.

The state was managed beautifully with Redux and the Redux Devtools web browser extension, allowing for rapid iterations and feedback where debugging would otherwise become very cumbersome. The useState hook was used profusely to separate concerns as well.

Keeping with React, the application was divided into logical sections that made it easy to add new features. Small components were created for different parts of the application, Redux actions and reducers were stored in their respective directories, containers and other higher-order components were given a special place in the frontend hierarchy, utility functions were well organized and accessible, pages and custom hooks also had their own place to reside.

Deploying with Docker remained a clean process, where a single Dockerfile was created to power the backend and another Dockerfile added for the frontend. Finally, all these instructions were put together with a readable docker-compose.yaml file, which purpose was to join all the different containers in a single place and to allow adding more microservices, such as Redis. Postgres was not used in this version for practical reasons when sharing the source code of the web application with others, but only a few steps would need to be taken to update Django’s settings file as well as pulling from DockerHub a suitable image to run Postgres.

Testing with Django went particularly well: the documentation had a clear guide about testing and any aspect of the application – serializers, views, models, forms, consumers, signals and so on – were tested according to the Django standards, leading to a whopping total of 119 passing unit tests, which are to be found inside a tests directory inside each Django application within the project.

What could have been improved

Planning of the technical stack could have gone much better. During the experimentation phase in the early steps of this project, good portions of functionality were implemented using Django templates without any frontend framework, except for the CSS framework Bootstrap. Features such as user notifications, personal messaging between two users with the ability to use different “inboxes” and “threads” was a very good selling point of the first version of this web application. Alas, due to time constraints, these useful additions were left behind (preserved with a Git history) but did not manage to make it to the final product. It would have been much more productive to take a closer look at the panoply of requirements to be implemented from the very beginning and determining the best fit as early as possible to avoid rewriting code from one framework to another.

Although Redux was a lifesaver in many circumstances, it was also a hurdle in terms of productivity because of its complexity and different moving parts. Being more careful to not use Redux at all costs everywhere possible, dispatching updates in the application would surely be simplified with the Context API using pure React or even some useEffect hooks, which were used profusely with good results. Immer was also introduced a bit later in the thought process when dealing with nested JSON objects. Although it made the remaining development easier, it was also probably a red flag meaning that the Redux store could have been simplified into smaller reducers where different selectors could access parts of the store that wouldn’t need to be nested objects. This may also indicate a flaw in the way models were implemented, but most likely there were errors made when dispatching messages in a part of the Redux store that really should belong somewhere else. In concrete terms, dispatching an event regarding users inside the “user” reducer was probably more complicated than simply dealing with all notifications in a different reducer. In actuality, this led to some dispatch actions that did more than one thing, which is not ideal when thinking about the concept of separation of concerns.

Although this is not perceptible in the final product, Django migrations were at times a pain to work with. This is because the models weren’t thought through very well at the beginning and despite the iterative process being an inherent part of software development, iterating over models in Django definitely counts as an exception to the rule where clearly defined models will not cause migrations to break or need to be modified in any way. It would have been more productive to produce well-designed diagrams, depicting the architecture of the application as well as the complete schema of the database to avoid surprises down the road.

State of the art in web development

On a technical aspect, many parts of this web application conform to modern web standards. The latest stable release versions of libraries were used whenever convenient and/or possible. This non-exhaustive list would include, at the time of this writing, React 17.0.2, Redux 4.1.2, WebSocket 1.0.34 and Material UI 5.5.0 for the frontend. On the backend, Django 4.0.3, Channels 3.0.4, Djoser 2.1.0 and Redis 4.2.0 were used.

When dealing with dependencies, Poetry was used with great benefit to make sure that Docker containers could be deployed reliably. Using Docker and Docker Compose was another touch that was added since most contemporary software will ship with a Docker solution for convenience. An excellent example in this regard would be PlantUML, a Java-based application to generate diagrams from textual descriptions that can be deployed in only a few seconds via Docker, launching a local server with the full capabilities offered by the piece of software in question. Nowadays, with Docker containers being more secure and very fitting for a deployment to the cloud at most big technology companies, it really is a best practice adopted by good software teams to work with containerized versions of their products.

The wider community was consulted in an effort to bring the maximum quality possible to the end result. Concretely, advice was taken from prominent figures in the web development world, including Dennis Ivanov and Laith Harb, both Django and React experts and Corey Schafer, an accomplished software developer from the Python world. Nigel Poulton – author of Docker Deep Dive – served as a good reference for the deployment part. Real Python was a godsend with its in-depth articles, for instance about debugging code with Pdb. Less well-known yet high-quality content producers inspired work with Djoser and the implementation of user posts, including Bryan Dunn and Legion Script. Cloudflare was also one of the best resources used to deploy a secure web server locally and to learn about JSON web tokens and their use specifically with Python. Needless to say, the official documentation from each major framework and tool was referenced all way through, many names which are going to sound familiar by now – Django REST Framework, React, Redux, Immer.js, Django, Djoser, Channels, etc.

Demo of the app

Resources and references

A retrospective on creating an impractical little tool just for fun

Sébastien Lavoie — Sun, 18 Sep 2022 20:06:00 -0500

Introduction

Building little projects to scratch one’s own needs is a great way to practice the craft of software development, or at least this is some consistent advice I have picked up from legendary figures in the field. Whether it be formulated by the famous John Carmack (an excellent interview with Lex Fridman was recently released) or a rising star like The Primeagen (he has a fantastic YouTube channel too), the core of the message remains the same: deliberate practice is required to make progress.

While I was in the middle of completing a BSc in computer science, I decided that a simple and reliable spreadsheet was not enough to keep track of my grades and decided to tackle the creation of a TUI – terminal user interface – to come to my rescue by providing, hopefully, correct answers. And so uol-grades-calculator (ugc for short) was born out of the desire to learn more about TDD in a practical way, the Hypothesis testing library written in Python, Sphinx and Read The Docs for publishing nice-looking documentation in reStructuredText format (the Markdown format being another option). Even though the goal of this tool has never been to gather a growing user base, it was a good learning experience because alternatives to it spawn up over time and, arguably, some of them were much friendlier and simpler to use (who would have thought that…).

Setting up the stage

To put things in perspective, this tool was never really under active development, but it did receive updates from August 2020 until September 2022, so the project slowly took shape in a time frame of a little over 2 years as can be shown in the following repurposed Gantt chart, where the length associated with each Git commit corresponds to the amount of time elapsed from one commit to the next.

Mermaid source of this diagram

Vim magic behind the scenes

While the chart itself is neither the prettiest nor the most interesting, the how of it has a possibly more entertaining story attached to it:

The list of commits was extracted with the command git log --pretty=format:'%s : %cs' > commits.txt to get a one-liner for each commit with the description followed by the date, piped/stored in the text file commits.txt.
Some Vim regex magic was done to substitute the first ‘:’ character on each line so that Mermaid wouldn’t struggle to render the graph: ‘%s/^$\w\+$: /\1 - /’. This pattern was actually predictable because the Conventional Commits specification was followed religiously right from the beginning.
The previous command left some ‘:’ characters in there that weren’t needed, so they were removed except where a digit was immediately following such that the last part of the line including the date would be left in place (here, \v is the “very magic” flag used so that we don’t need backslashes around the captured group, i.e. the digit, which looks a bit more readable): ‘%s/\v: (\d)@!/ -’.
To show the elapsed time between two commits, a Vim macro was concocted so that the date on the current line would be copied and pasted on the following line in a specific way, which works because the Git history appears in chronological order by default: ‘kf:wv9ly0jA,^[pj0’. It’s Ugly but it gets the job done ;).
- In essence, it goes up by one line (k), searches for the : character with f (which precedes the date), moves ahead by one word, highlights the date which is in the format YYYY-MM-DD with v9l, yanks/copies the date with y, goes back to the beginning of the line (0), goes down one line with j, Appends a , at the end of the line, goes out of insert mode with a Ctrl - c shortcut (which visually translates to the escape sequence ^[), pastes the date, goes down one line again with j to be ready to process the next date and goes to the beginning of the line again with 0 so we don’t miss a match. Vim macros are definitely quite expressive in a terse way!
- Then, it’s just a matter of running the macro for the number of lines below the cursor in the file. We can quickly get a sense of the number of lines in the file with Ctrl - g and run the macro on the required number of lines with 100@a, replacing 100 with the number of lines and a with the register where the macro was saved (in this case, in the register a with qa to start recording the macro there, which is to be stopped by pressing q once more in normal mode).
Once the macro is run, the output needs to be reversed so that the initial commit can be displayed where the graph starts in the top-left corner, which can be done with the clever command ‘g/^/m0’: it runs on every line of the file, successively putting each line at the very top of the file (line 0) to have the effect of reversing the whole document!
Finally, the graph is generated literally by copying and pasting that list of commits into a Mermaid textual chart, which shall be the topic of another post!

The whole process is done in a few minutes at most, making it a nice solution to deal with manageable outputs under a few thousand lines. Otherwise, it might be more convenient – albeit a bit more time consuming too – to write a script to do the work since Vim macros, when run at a relatively large scale, can be slow and error-prone.

What went well

With all that fluffy preamble, we might hope that something went well in the end. Well, there were a couple of noteworthy things indeed:

Even though I worked on this project alone, I created a bunch of GitHub issues that I assigned to myself, each with its own set of tasks, acceptance criteria, a short analysis section as required, linked pull requests and referenced commits. It is more work and arguably could be done more simply in a notes application, but it feels good to be publicly accountable for one’s work and that also encouraged (very few) people to submit their own issues. Additionally, managing tasks on a platform like GitHub makes it easy to review the state of a project and to manage a backlog of tasks, which was a nice plus when the project was linked to what was meant to be its successor, uol-grades-calculator-server, a backend using ugc as its API which was to be hooked to a React frontend to display the data (more on that in the list of things that went poorly).
The TDD approach was embraced right from the start and I consider that being one of the highlights of the project in hindsight. With over 250 tests (some of which are randomly generated to do property-based testing with Hypothesis), I felt confident it was giving somewhat sensible answers.
Best practices were adopted: Pylint, the king of complainers amongst the Python linters, was very verbose but set up to be slightly more quiet after a while. A pyproject.toml file was included, a clean pytest config file was used to automate the input of some parameters when running the test suite, the project was developed with the intention of being deployed as a package, only popular and stable libraries were used and just as importantly, only portable formats like JSON made it to the final version of the project.
Sub-modules were created to make everything a bit more… modular.
A GitHub workflow was added to automatically deploy new releases to PyPI.
Documentation was generated and hosted on Read The Docs, with clean outputs and images and an entire section dedicated to developers (i.e., myself :)) so that reproducibility wouldn’t become an issue for any step taken along the way.
Using pre-commit helped with catching some annoyingly formatted code before reaching production by taking advantage of Git pre-commit hooks.

With what seems like a glowing review so far, needless to say, a whole lot went wrong too.

Illustrating the summarize command.

What could have been done better

Independently of the expectations with regards to the user base, it would have been beneficial to do more research and planning initially before writing a single line of code. After working on other projects, I’ve found that what works well for me is to follow a specific set of steps, broadly speaking:
- Gather a list of requirements as precisely as possible to validate assumptions and define the correct scope for the work to be done (YAGNI!);
- Produce some early flowcharts to make sense of how the application might be structured and iterate on this until a clear winner emerges;
- Produce low-fidelity wireframes (even for a TUI, that would be informative before committing to doing the wrong thing!) – even if this is just quickly sketched on paper;
- Create high-fidely mockups (or at least reasonable-looking ones for a small project like this one) once things have been thought through carefully;
- Design a general plan of work before writing any code to have a better idea of how the different parts of the system should communicate together;
- Research existing tools to be leveraged so as to not re-invent the wheel all the time and see how they fit in together;
- Go with TDD and stick to it piously for anything of relative importance that should be tested;
- Go at it with atomic commits so that rewinding errors on the path will be a breeze – at least that ought to help significantly.
Related to the first point, issues could have been more substantial instead of having to often deal with useless refactoring due to not thinking things through fully.
With more mental processing done upfront, it would have been clear which features should have been part of the tool earlier and which ones could have waited longer. Glancing at the Gantt chart above, one can see that many commits happened at the beginning just to get the structure right, the project was not documented until much later and basic functionality such as being able to plot results (which was really a core feature to be expected) was not integrated until past the mid-life of the project.
On top of the above, it was clear that using YAML in the presence of the other technologies used for a backend and frontend to the CLI made things more complicated and it wasn’t until one of the latest versions that JSON was used instead.
The user experience could have been improved drastically at the beginning by relying on well-known libraries such as Rich to make the UI much more appealing and even a bit more interactive.
Some features were presented in a way that did not make much sense. For instance, the Dockerfile does allow one to use the tool from a Docker container, but not all functionality works out of the box that way (e.g., saving a plot to the filesystem or loading a configuration file).
It wasn’t productive at all to start scattering the few resources put into this project by creating a sister repository uol-grades-calculator-server. The CLI should have been more feature-rich before considering such an endeavor and it should have been made in a way that’s easier for a backend to consume.

Showing the image generated by running the plot modules (sub-)command.

Key lessons learnt and where to take it from here

Studying better the available libraries would have avoided some headaches. For instance, the display of what should have been tabular data started out as plain JSON (printing a dict object), then beautifultable was used to create some colored output, then pandas replaced the previous library because some limitations were found with it – reverting to a plain table output in the process – and finally rich was used to create a more decent-looking output.
Likewise, spending more time in the planning department before getting started would probably have resulted in a tool that’s more straightforward to use, focused on the most needed features and possessing a broader feature set by not having invested extra resources on building a backend and a frontend separately. It should have been either a more powerful TUI limited to be used within a terminal or a web application, but not both in the way it was designed.
Work on small, isolated parts of the system. Some issues were split into large chunks of work (this one for instance, implementing at once the plot command and its sub-commands). This wasn’t a problem per se given that I worked alone, but it would be harder for someone else to follow and to create a reasonably-sized pull request from such an issue. This feature could have been broken down into different sets of related flags.

A simple component diagram demonstrating how the CLI was to be used in a greater context.

Conclusion

“Regardless of what we discover, we understand and truly believe that everyone did the best job they could, given what they knew at the time, their skills and abilities, the resources available, and the situation at hand.”

--Norm Kerth, Project Retrospectives: A Handbook for Team Review

All in all, this has been a fun experiment to bring to fruition. It would have been much quicker to use a spreadsheet given what the final result ended up being, but the experience obtained was worth every drop of developer sweat! Using Python felt appropriate as iterations of the tool could be produced fast and the CLI was responsive enough to be useful and pleasant to interact with thanks to its intuitive commands. In the near future, using a library like Textual could be more rewarding as one could expect a much nicer visual experience in the terminal at a fraction of the current implementation efforts.

Resources and references

What it took to propel a career in tech in five years

Sébastien Lavoie — Sat, 18 Jun 2022 14:22:00 -0500

After having kept a concise (and incomplete) log of my technical learnings for five years at this point, I can see I’ve come a long way, yet this is only the beginning. From now on, it is time to focus on taking concrete action, build on the foundation that has been laid out over time and start appreciating life from a broader perspective, too, as there are of course many aspects that have contributed to put me at this exact spot on this journey. Theory leads to a deeper understanding in any field, but in the end, the results being produced are what matter most and spending quality time getting good at one’s craft is key to making real progress. I think the most important steps to take in this context are getting into the habit of learning constantly and working towards generating a profound desire to excel, just for its own sake.

To glory and beyond!

The output below was formatted automatically with a tool that converts the content of a Google Sheet into Markdown. If you’re curious, you might enjoy reading this article!

2022

April

Articles
- Google Cloud Platform
  - Access control lists (ACLs)
  - Set up OS Login
- React
  - JSX In Depth
- Technology
  - Flutter is better than React Native
- Web development
  - How I built a modern website in 2021
University of London
- 3D Graphics and Animation
  - Computer Graphics: From Pixels to Programmable Graphics Hardware (book)
    - Lighting models and shading
    - Transforms in 3D
  - How to set up a Unity project in GitHub
  - LEARN UNITY - The Most BASIC TUTORIAL I’ll Ever Make
  - Unity
    - Orientation
    - Coordinates
    - 3D models
    - Components
    - Lights, Prefab
    - Essential Unity Concepts
    - Basic movement, building a scene
    - Types of light
    - Script communication
    - Physics and collisions
    - Movements and forces
- Mobile Development
  - Adaptivity and Layout
  - App Development
  - Core Components and APIs
  - Dark patterns
  - React Native Express
    - JavaScript
    - React
    - Core Components
  - React Native flex
  - React Native – Programming UI elements
  - React Navigation
    - Read documentation
  - Skeuomorphism, minimalism and neumorphism
  - Universal Access in Human-Computer Interaction (book)
    - User Interfaces in Dark Mode During Daytime – Improved Productivity or Just Cool-Looking?
    - Impact of Sentence Length on the Readability of Web for Screen Reader Users
  - Wireframing
YouTube
- Bash
  - bash quoting is really not that difficult!
- Git
  - Git Annex Is The Coolest Program You’ve Never Heard Of
- Graphic Design
- NFTs
  - You’re Wrong About NFTs
- React Native
  - React Native Tutorial for Beginners - Build a React Native App
- Software design
  - How To Avoid Big Upfront Design
- Software engineering
  - What Software Architecture Should Look Like
- TDD
  - How To Perfect Software Design With Test Driven Development
- Unity
  - Learn Unity - Beginner’s Game Development Tutorial

March

Articles
- Data science
  - The Complete Guide to Time Series Analysis and Forecasting
- Django
  - How to Create a Custom Django User Model
  - Writing your first Django app, part 5
    - Testing in Django
- Docker
  - Networking with standalone containers
- JavaScript
  - Immer.js 101 — Making Redux state updates in React predictable
- Security
- Streamlit
  - Visual time series forecasting with Streamlit Prophet
Documentation
- Django
  - Signals
University of London
- Databases and Advanced Data Techniques
  - Review course concepts (see past entries)
YouTube
- Django
  - Build an Asynchronous Chatroom with Django and Channels
  - Django & React Authentication
    - Password reset
    - Sign Up & Account Activation
  - Docker - Django, Celery & Redis Docker Compose setup
- React Native
  - Build an Instagram Clone with React Native, Firebase Firestore, Redux, Expo

February

Articles
- Python
- Terraform
  - The easiest way to run a container on GCE with Terraform
Books
- Terraform
  - Terraform in Action
    - ch. 1
Courses
- Google Cloud Platform
  - Google Cloud Platform Fundamentals - Core Infrastructure
- Python
  - Build Web Apps in Python with Streamlit
- Terraform
  - Implementing Terraform with Google Cloud Platform
  - Terraform - Getting Started
Documentation
- Google Cloud Platform
- Terraform
  - Get Started - Google Cloud
University of London
- Advanced Web Development
  - Learn React & Django - Chatroom Concept - React Django+Channels Chatroom Heroku Deploy
YouTube
- Blender
  - Blender 3.0 Beginner Donut Tutorial
    - whole series
- Django
  - Building a Social Media App With Python 3 and Django
    - Replying to Comments
    - User Notifications
    - Adding Images to Posts
    - Sending DMs
    - Finishing DMs
    - Sharing Other Posts
    - Tags
    - Finishing tags
  - Django & React Authentication
    - Navbar and check authenticated status
  - Django & React Tutorial
  - Django + React Notes App
  - Django Channels & WebSockets Oversimplified
  - Django REST Framework Tutorial Class Based Views Pt. 1
  - Django REST Framework Tutorial Class Based Views Pt. 2
- Python
  - Python lists remember what you did to them

January

Articles
- Docker
  - How to SSH into a Running Docker Container and Run Commands
  - How to squash Docker image
- Mastering the craft
  - What Makes a Senior Developer… Senior?
- Python
  - Logging in Python
- SSH
  - How to access a Linux Server using an iOS device
Books
- Docker
  - Docker Deep Dive
    - ch. 1-6 (Docker engine, images)
    - ch. 7 (Containers)
    - ch. 8 (Containerizing an app)
    - ch. 9 (Deploying apps with Docker Compose)
Documentation
- Cloud
- Django REST framework
  - Schemas
- Python
- Slack
  - Web API methods
University of London
- Advanced Web Development
  - Asynchronous Web Services (Celery, Django, Pillow)
  - Attacks on websites (XSS, CSRF, SQL injection, clickjacking)
  - Consuming WebSockets
  - Deployment automation
  - Django authentication and administration
  - Django channels
  - Introduction to production deployment
  - Introduction to web sockets
  - Known Issues and Best Practices for the Use of Long Polling and Streaming in Bidirectional HTTP
  - OpenAPI
  - Profiling and performance
  - Scaling and load balancing
  - Tutorial Pillow (Python)
  - Web APIs
- Databases and Advanced Data Techniques
  - Distributed Relational Databases
  - Document databases and MongoDB
  - Introduction to Linked Data and the Semantic Web
  - Introduction to semantic databases
  - Linked Data in a Graph Database
  - Multimedia databases and Information Retrieval
  - Using XML
  - XML Schemata
YouTube
- Django
  - Building a Social Media App With Python 3 and Django
    - Landing Page and User Authentication
    - Posts and a Social Feed
    - Get a Post and the Comment Model
    - Edit Posts, Delete Posts, Add Comments
    - Profiles
    - Followers
    - Likes and dislikes
    - User search
    - Final improvements Part 1
    - Final improvements Part 2
- Linux
  - Linux File System Structure Explained

Book summary: Letters to a New Developer

Sébastien Lavoie — Mon, 08 Nov 2021 10:12:00 -0600

Introduction

I’ve taken quite a few notes while devouring that book, which is a quick and easy read. It’s not the typical non-fiction, technically-oriented manual: this book makes you feel like having an older family member taking the time to sit right next to you to openly share all sorts of golden nuggets from a lifetime of rich and varied experiences so you can get prepared to face the challenges that lie ahead in your own career and avoid some of the more common mistakes during the journey. It is the kind of publication you can read literally five minutes at a time thanks to the convenient format it adopts.

Some of the following notes are verbatim and reading the whole title will be worth it! Nevertheless, you will find below some passages that deeply resonated with me, reflecting the gist of this prose from my perspective. Beware! There are numerous gems to be gleaned from this excellent author.

Book summary

First month at a job

See problems as opportunities to contribute.
During the onboarding, figure out:
- All the HR stuff: benefits, agreements, bonuses, interacting with different departments, meetings, requesting vacation, raises/reviews cycles.
- And the technical stuff too: who to ask questions to, how to ask them (batch them, chat, meeting, etc.), how long to work on a problem before asking for help, how to communicate progress, how to set up a local development environment, CI/CD and branches process, when do you know you’re done working on a task, how are tasks managed (manager, issue tracker, etc.).
Questions that arise and have no answers → See where to document them (e.g. wiki).
First impressions:
- Be there early.
- Say what you’ll do, then do it.
- Research before asking questions (and record answers).
- Volunteer to take extra work (don’t become a punching bag, though).
- Own your mistakes but don’t make the same ones twice.
- Be polite and professional, always.
- Make sure with manager we’re on the same page, working on the right stuff, at the right cadence with appropriate updates.
- Write docs to make the next hire easier.
- Have the reputation to be a hard, smart worker: it will follow you.
Be ready for the trepidations: celebrate successes ("today I [fixed, learnt…]").
To excel:
- Communicate what you’re working on.
- Ask questions.
- Don’t make the same mistake twice: write down what went wrong and plan to avoid repeating them.
- Show up consistently.
Know the team: their names, their roles, their behaviours (who posts what, who is in which meetings, who seems funny and approachable on Slack, etc.).
Read code and take notes: dig into a section of the system, follow data flow, make diagrams of how things are connected.

Questions

Ask prepared questions:
- Research before asking.
- Find where to search (Slack, issue trackers, YT, internal docs or wiki, etc.).
- But also be mindful of how long you should spend time searching (Is there a tight deadline? Are you wasting your time?).
Ask good questions:
- Specify the problem in detail, narrow the scope as much as possible.
- Show you’ve done research. Links, videos, posts, logs (without sensitive info), etc.
- Follow up after a day or so to share additional research.
- Express gratitude.
- If you have the answer, add it.
Don’t be afraid to ask questions:
- If it’s a learning experience, spend more time figuring out the answer.
- The more time you have before the task is due, the more time you should spend looking for an answer.
- How busy is the rest of the team? If they can’t help, you’ll have to do it yourself.
- If there’s a bug that’s really specific to the internal project, asking for help from coworkers is probably best (be careful not to disclose business knowledge!).
- Have you tried working on other aspects of the task? It may unblock you.
- Put in the effort before asking: research, list hypotheses, etc.
Ask why:
- It may help prevent mistakes down the line (e.g. wrong choice of tech) and get more perspective, understand better.
- Document findings in wiki or similar (could be in a question/answer format).
If you don’t know something, don’t guess or make an incorrect decision: admit it and say you’ll find out the answer (may have to ask others in the process too).

Writing

Read your texts out aloud: spot typos, awkward sentences, etc.
Document the intent behind what you’re doing (code, reason for change, who asked what…).
Emails:
- As short as possible. Otherwise, add executive summary at the top.
- Is it sensitive? Maybe have face-to-face instead.
- Add links to supporting documents rather than attaching them. Email chains are hard to follow: may put them in a document instead.
- Don’t use relative time references.
- Keep them focused: go into one topic. New topic = new email with different subject line.
Real-time messaging: Default to public channels to benefit the organization.
Write a technical ebook: working on a complex project over time and being able to see the high-level and details, all presented in a coherent way for the readers.
Write and update technical documentation: you save time in the long run, it’s useful for you and the rest of the organization.
Blog: this is the process of clarifying your own thoughts. You’ll be a better writer and thinker for it.
Motivation about writing:
- It crystallizes thoughts, make them clearer;
- Builds credibility;
- Helps others;
- Illustrates an ability to convey technical information and context.

Tools to learn

Leverage for increased productivity:
- Test suite: living documentation. Can evolve code base without fear.
- Libraries and frameworks: saves time.
- IaaS (infrastructure as a service, e.g. AWS EC2): can use APIs to manage.
- Paas (platform as a service, e.g. Heroku): Allows to focus on business logic.
- Saas (software as a service, e.g. Google Apps): integrate with existing software.
Command-line:
- Typing is quicker than using GUIs for many tasks.
- Can automate recurring tasks easily.
- Can easily share task scripts with others.
- jq, awk, sed: put them to good use.
Version control: if in doubt, use it, except for large files and other scenarios that don’t make sense.
Text editor: learn one well.
IDEs: Use them when it makes sense (programming in Java with a text editor? Probably a bad idea).
Standard library: get a good overview of what’s available, leads to transferable skills, more idiomatic code, better tested, etc.
Automated testing:
- Prevents bug regression.
- Serves as live documentation.
Network engineering: Learn about it, it will pay off. Know:
- Basics of routing;
- Basics of DNS;
- Rundown of the OSI model;
- TCP/IP, HTTP, what a proxy is;
- Answer questions like How do CDNs work, what’s the difference between HTTP and WebSockets, how does SSL work, what can nginx be used for…
SQL: this is fundamental knowledge as it’s almost everywhere.
Debuggers: learn how to debug in your IDE, on the command-line and in a web browser (your IDE won’t always be available).
Benchmarking: do not optimize prematurely and use benchmarking tools when performance matters to see if the code is fast enough.
Search engines: learn to be an expert Googler.

Practices

Solve problems, don’t just write code:
- Use library/framework.
- Use third-party SaaS tool.
- Make sure tasks need to be done. (Why are you doing it? Does it provide value?)
- If a task isn’t recurring often enough, manual work may be the best way to go (time to automate vs. actual savings).
Think ahead, envision all paths (not just the happy one) before taking a decision.
Read code: scan it; dive down; use the scientific method to make changes and debug.
Get good at estimating your work:
- Opportunity cost (can’t do everything: prioritize and do the tasks that bring the most value).
- Scheduling: software is not built in a vacuum, hard and soft deadlines will occur.
- Discussing requirements first can save a lot of troubles and money: building the wrong thing quickly is of no use. Use lightweight prototyping tools instead of building the whole software.
- Recommended reading: Software Estimation: Demystifying the Black Art, by Steve McConnell.

Here’s one way to keep track of your estimates in a spreadsheet:

Task	Low estimate	High estimate	Notes, questions, research	Final estimate	Estimate in days
Requirements definition, research, development, testing, deployment, estimates on bug fixes…	Everything going smoothly.	Facing obstacles, having to rebuild part of the system, etc.	Answer questions and verify assumptions to complete the estimate.	(low + high) * 1.2 (add 20%, “fudge factor”)	Total hours estimated divided by number of ideal hours per day

Tracking the actual accuracy of tasks can be insightful too.
Always tell people who are depending on the completion of a task if it’s going to take longer than expected. Share alternative solutions if you have any.
Even for side projects or when working on open source projects, it may be useful to practice estimating (and users like seeing a roadmap).
Debug systems:
- Make the problem as simple as possible.
- Begin with an hypothesis and either prove it or refine it.
- Determine the desired end state: fix the root problem but if it’s too costly, find a workaround if possible.
- Pay attention if anything seems amiss.
- Keep notes about what you’ve tried.
- If there’s a new bug, inspect recent changes in the log.
- Write an automated test before fixing the bug so it doesn’t reappear and you’ll be sure when the bug was fixed.
- Follow the flow of data (e.g. in a three-tier application, start with the browser or with the database).
- Minimize impact for users if the bug is in the production environment. Ideally, test on the staging environment.
- Test the right thing. Make sure that what’s being tested mirrors as closely as possible where the bug appears.
Assume positive intent: be solution-focused.
- ROI on trust: about 4% of the population are sociopaths; meaning 96% of the population has some working conscience and could be trusted by default to reap great benefits. This may speed decision making, for instance.
Express gratitude: it will make you feel better and showing appreciation to others will make working with colleagues a breeze. Thanking people and writing a gratitude journal will be helpful.
Cultivate the skill of undivided attention:
- To produce your best output, you need to commit to deep work.
- To thrive in today’s economy, you need to be able to quickly master hard things and be able to produce at an elite level in terms of both quality and speed.
- To learn hard things quickly, you must focus intensely without distraction.
- Add routines and rituals to your working life to limit the amount of willpower required to get started.
- Book recommendation: Deep Work: Rules for Focused Success in a Distracted World, by Cal Newport.
Build empathy: Remember your own frustrations. Your users are people who can struggle on tasks you’d find simple yourself. They’re trying to make something work, just like you.
Don’t complain about the code:
- It’s not helpful.
- It displays a lack of empathy.
Avoid jargon: If you can’t explain it to a non-technical user, you don’t understand it well enough (good framework: try the Feynman Technique). You gain clarity for yourself, the ability to teach others and you can influence your organization. (Personal additional: get good at critical thinking and use frameworks for better thinking).
Time is money: buying services from others gives you time.
- Buying a book or video instead of reading free documentation.
- Buying and using exceptional tools (e.g. JetBrains IDEs).
- Paying for support.
- Buy commercial software.
- Pay for consulting or training.
Say no:
- If there are pending tasks, ensure they are worked in the right order of priority.
- Life is short: if you end up working 90 hours per week, make sure you’re happy doing it and are not sacrificing other important aspects.
Build on your own: pick a project you can envision yourself sticking to for more than six month and learn.
Consistency is key: learn by showing up every day. Becoming great at something takes time.

Understanding the business

Software is about people, not code:
- Software is created for people.
- Users need to be heard to buy in.
- Most people don’t care about the code. Their goal is to get things done. Beautiful code that doesn’t solve the right problem is useless.
Outcomes over output: the end-goal for the business is more revenue, more profit, more users, more product availability, happier users, etc. Your contributions should be holistic in nature and work towards achieving these goals.
Understanding the big picture of why money is spent on certain projects or services pays off as you can better contribute to the bottom line of the company by knowing why things are done the way they are.
Business model: know how money is being made. Get a good understanding of the domain in which the business operates.
Knowing how software is used at the company helps you to understand the business.
The size of a company impacts how it solves problems: a bigger company will usually move more slowly towards its goals when change is required, attacking possibly bigger problems with powerful tools and teams. A smaller company may pivot more quickly and shift focus, but it may not work on problems of comparative scales.
Starting a company:
- As a new developer, starting one wouldn’t be wise. Learning to become a software developer and learning how to run a business all at once can lead to a huge amount of stress.
- It will take longer than you think.
- Know your financial runway (“burn rate” vs. what you have in the bank). Extend it by lowering the burn rate.
- Consider your emotional runway: how are things going for you outside of work? Are there any trying events like moving or having a new baby?
- Talk to your customers: learn from them. Get feedback.
- Provide value to customers: you won’t have time to make everything go as smoothly as you’d like, but the customer must still be served.
- Know your market.
- Be ready to move away from code and focus more on a position of management as the company grows.
Learn from your customers:
- Digging into a ticketing system will be informative to understand how a product is being used and may reveal flaws or bugs.
- Focus on the problems mentioned, not on the solutions proposed.
- Ask them about their pressing problems, not only those that have been reported so far. This may inform about issues with related systems.

Learning

Never stop learning:
- Be clear about why you want to be learning.
- Once you have the why, determine the what that will take you there.
- Once you have both of the above, find the how.
- Execute.
Build expert intuition:
- It’s the difference between a junior and a senior developer.
- A regular world + many opportunities to learn + frequent feedback + expert intuition.
- Regular world: gives a chance to find out whether you’d prefer to specialize in a specific branch of the software development tree or remain a generalist.
- Many opportunities to learn: write code, work on technical projects and observe those who are more senior than you.
- Frequent feedback: code reviews and one on ones with the manager. Reflect on your performance with a daily journal.
Know what to learn (i.e. general skills vs. specific technology and techniques):
- Domain knowledge: understand the business you are in, who the major players are, etc. If you stick in a given domain long enough, you become more valuable as this kind of knowledge tends to last a long time. Talk to experts and read books.
- Theoretical expertise: data structures and algorithms, HTTP and best practices fall within this category. It serves for years as a solid foundation for everything else. Books are a fine source to delve deeper.
- Practical knowledge:
  - React, Kubernetes, Rails are some examples of specific technologies that belong in this category.
  - Have a good understanding of the underlying technology whenever possible (e.g. learn the language in depth, not just the framework).
  - This type of knowledge tends to age quickly and can be learnt through videos, conference talks and tutorials.
- Leadership knowledge: This will pay dividends through your career. Books and experience in the real-world are useful here.
Avoid being an expert beginner (an authority with nontransferable skills):
- Never believe you have all the answers.
- Don’t bend tools to do something they weren’t meant to do.
- Don’t ignore best practices (but don’t blindly apply them everywhere either!).
- Keep in touch with the larger software community: you may temporarily be at the top somewhere technically speaking, but that won’t remain true in a bigger context.
- Pair with people that are more experienced than you: ask them questions, read the pull requests they submit.
Pattern match to be a just-in-time learner: explore by pattern and analogy (e.g. compare terminology that’s similar to tools you’ve used before) and learn just enough to get the task done. Learn the nuances as you go.
Learning boring stuff:
- Focus on the big picture. Why are you learning this?
- Notice the fun parts.
- Take breaks when needed. If no strict deadline is in sight, you might cross off a different task on your to-do list for now.
- Automate what you can but consider clearly the time savings when doing so.
- Learn relevant things: this applies once you have a base to lean on, but accumulating random concepts that aren’t often applicable after that isn’t useful (unused knowledge rots and gets forgotten).
Your team will teach you:
- They can teach you more than you can learn on your own: tools; languages; frameworks; business domain knowledge; approaches to problem-solving; understanding of the problem; stakeholder empathy.
- Observe how senior people work (face-to-face, read their code and pull requests).
Use an RSS reader: for blogs, online community discussions, tags in Stack Overflow, news sites.
Listen to podcasts: good to gather ideas about high-level concepts while doing other things (dishes, exercise, watering plants, cooking, etc.).
- Listen to those that are domain specific to help you at work.
- Technology specific: expose yourself to tools, techniques and libraries.
- General software development: those are broadly applicable to software engineering and will work across different jobs you have within the industry.
Subscribe to email newsletters: great to get curated content on a given topic. These could include software development practices, security, AWS and career skills.
- Find them with searches like <subject area> weekly/email newsletter.
- Read the archives first to get a sense of whether you’ll enjoy the content.
- Like with podcasts, focus on what’s interesting and skim the rest.
- When sources lead to more articles of interest, those can be captured in your RSS reader.
- Good to explore new technologies and frameworks.
- To start your own, TinyLetter is a great option.
Read great books about software development: don’t look for cutting-edge information but rather for a source of timeless practices.
- Good examples include The Mythical Man-Month, The Pragmatic Programmer, Refactoring.
- Ask great engineers you respect for recommendations.
Listen actively:
- Do not multitask.
- Take notes of… salient points; ah-ha moments; terms to review in the future (those concepts you may not understand). Note the date of the conversation as well.
- If it is a video chat, see whether it can be recorded. Turn on video if possible to get contextual clues.
Learn two programming languages:
- Lets you see the strengths and weaknesses of your first language.
- May support concepts that weren’t there before (e.g. classes in Java vs. their absence in Perl).
- Makes it easier to learn a third language.
- Illustrates different approaches to common problems.
- Teaches you how languages fit certain problems better than others.
- Makes you less passionate about your first language. Programming languages are only tools.
- Makes it clear what you understand about language #1.
- Tutorials are great to get started and side projects even better to cement the learning.
- Generally, learn at least two of “everything” (e.g. databases, frameworks, development methodologies, etc.).

Mistakes

Get used to failure: learning something new is difficult. Document internal knowledge as best as you can so the same mistakes do not repeat themselves. Look around (online, colleagues…) to make sure the problem you are trying to solve hasn’t been worked on already and take any new information into account.
Making mistakes is okay:
1. Find out what the oversight was so you don’t repeat the same mistake.
2. Acknowledge you made an error.
3. Clean up the mess, with or without help.
4. Avoid making the same mistake again. You might keep notes around, write a blog post, etc. Document it or create a script to solve the issue from now on if applicable and share your new knowledge with the rest of the team.
Mistakes are forgiven, hiding them is not:
- Make sure you did a mistake to start with.
- The people who need to know about your errors should be told. Transparency and honesty are important.
- Think of a plan to fix the mistake (both short-term and long-term).
Don’t make the same mistake twice:
- Make sure you understand what the mistake was.
- Dive deeper when it makes sense. For instance, it is worth mastering a version control system as this is a tool that’s going to be used all the time and is critical to understand when working with other software engineers.
Don’t be afraid to “fail”: the only true failure is to quit working towards success.
- Failures don’t stop happening when you become a “better” developer.
- The more you fail, the more you learn.
You don’t know what you don’t know. Doing pattern matching and recognizing concepts at a high level will help when there’s a need to dig deeper.
When you see someone else making a mistake:
- Make sure you have all the necessary context (you usually don’t have it) before jumping in to shine a light on how you think things could be done or fixed.
- Make sure you fully understand the issue at hand.
- Think about the ramifications of a poor decision: the higher the stakes are, the more likely it is that you should/could intervene.
- Contemplate your role: depending on the relationship you have with that person, tolerance levels will change (personal note: ask questions early as needed, but make sure you show progress as stagnation won’t lead to joyful relationships with your colleagues and may get you fired).
It’s OK to fail and it’s OK to feel bummed about it. See how core/important the task is for the company, whether somebody else would be better positioned to do it, how long it would take you to become good at it, whether you enjoy it, whether there’s another way to solve the problem, if there’s someone else at the company who could teach you. Don’t beat yourself up with despair: be strategic about the way you spend you efforts.
Admit your weaknesses… and own them.
- Is it something innate, like bluntness, or is it something you can learn, like Python?
- If a weakness is related to a core competency needed for your job, be proactive in overcoming the obstacle or find another job that’s better suited to your skills.

Your career

Favor learning over earning: take the job with the highest learning potential.
- Beware of stunting your growth by working alone.
- You will become more valuable to your future employers and have a better idea of what you like.
You will never be in a better position to leave a bad job than before you start: interviewing is a two-way street.
- Do your research before accepting a job offer.
- Investigate through your network, e.g. LinkedIn, if you have a connection that worked at that company and ask about the good and bad of this organization.
- Ask good questions about the company during the interview.
- As an exception, you might want to consider getting your feet wet in the industry with your first job by being less picky, especially if the job market is bad. Experience will prove useful.
Pick a flaw: no company is perfect.
- You don’t want an indifferent team just punching the clock and waiting for the week to be done.
- Avoid nonsense: ask how a typical day goes, who makes decisions about priorities, in what kinds of projects does the company shine and what the best and worst parts of a job are.
Preparing for a recruiting event. Have: a resume; a good intro to yourself; a good conversation you can have; be well presented.
- Resume: make sure you have the address, phone, email, GPA and graduation date (for students) at the top. It should show you’re motivated, skilled, passionate, adaptable, collaborative and articulate. When you have less experience, projects should be the centerpiece of your resume. Project descriptions should be brief and focus on the what, why and the outcome, avoiding mentions of tech stack unless it adds to the narrative. These can be side hustles, open source, personal interest or hackathons. Have more than one resume for different audiences.
Practice programming: you learn by doing.
Start at a consulting company if you’re looking to gain experience across technologies, domains and businesses. Work for a small one if you want to have an impact. This will provide contacts and set you up for future work. Seek out mentorship, request conference attendance and pursue assistance as formal education programs tend to be lacking in those environments.
Potential vs. delivery: Ask smart questions, show you’ve learned stuff. Say what you’re going to do, then do it.
- The more experienced you become, the more you are assessed on your ability to deliver.
- Expect to demonstrate your experience during interviews.
- Periodically, or when you complete a project, take a few minutes to write some lessons you’ve learned.
- Take risks early as pivoting later on in your career becomes more difficult.
- Book recommendation for more senior developers when switching domains, tech stacks or employers: What Color Is Your Parachute, by Richard N. Bolles, with Katherine Brooks.
Maintain work-life balance: preserve a firm boundary. Work is a marathon, not a sprint.
- Get clear on how the employer expects you to handle unplanned work with conflicting priorities.
Listen to the voices when they help and ignore them when they don’t:
- Be willing to do things you didn’t think you wanted to do.
- Connect with non-tech people outside of work to take a break and experience life from their point of view too.
Manage your career: know your goals; communicate your goals; progress towards them.
- Knowledge: pick an interest and follow it. Plan for a decade or more, with some milestones in between.
- Communication: communicate your desires and ambitions. Good companies want to see their employees grow.
- Progress: study for a certification, read a book, go to a meetup. Be active.
Know your runway: this is the time you have at your current spending until you have no money. Calculate it once a month and know how long it would take to get a new job (guess).
- Get a job before you need to to avoid accepting an ill-fitting job due to desperation.
- Cutting expenses is hard but it’s where you usually have the most control.
- Spending savings for a career transition is an investment in yourself.
- Activities that draw down your savings should lead to new opportunities and valuable personal knowledge.
Managing one on ones: for remote ones, it’s best to have a great Internet connection, a quiet space and video turned on (personal note: you do lose visual clues without video, but you can connect deeply without anyone being too self-aware with voice-only communication).
- Regular ones (weekly or bi-weekly) are preferred so you don’t dread feedback only when you ask for a raise or do something wrong. Knowing your manager beyond these scenarios will help.
- Reasons to schedule them: understand where we are headed so I know what to learn; know how I can best help the manager and the company; understand your priorities better.
- The direction of the one on one should be determined by the employee.
- Shoot for regular ones that do not get re-scheduled. Otherwise, propose having them less frequently if needed.
- If one-on-one meetings aren’t desirable, go for an asynchronous weekly status report and get feedback from your manager this way: it is critical for your career growth early on in your career.
- Come with a prepared agenda to make the most of them. A shared document in reverse chronological order works well. Example questions to add:
  - How should I have handled situation X?
  - I would like to learn more about Y; what are the opportunities?
  - I’m struggling with <problem>, do you have any suggestions?
  - What are the challenges you see facing our team during project Z?
- You should be able to trust and rely on your manager/boss. If not, improve the relationship or go elsewhere: being constantly under threat and not knowing what to expect is mentally draining.
Write a brag document: show everything you’re proud of so they know you’re a great team member. It should be thorough yet easy to read. Along with the agendas from your one-on-one meetings, it will serve to highlight your accomplishments for your next performance review.
Be adaptable and authentic.
Working remotely: you need a fast Internet connection; iron discipline; be okay with solitude and have the ability to work through communication obstacles.
- If you are comfortable asking a dumb question, remote work can work well for you.
- You will need to get used to asking questions and interrupting others as you more often than not won’t have enough context to infer whether it’s a good time to ask or not. It’s better than the alternative of wasting your time.
How to go through a layoff: this isn’t personal and you will get through it.
- Ask questions:
  - Know the deadline to sign the necessary documents;
  - Find out if there’s a severance and how much is it;
  - What about funds like 401k, FSA or HSA;
  - How can I say goodbye to teammates (emails and LinkedIn are good options);
  - What about company’s property, e.g. laptop, books and equipments;
  - Who can I contact if have I more questions;
  - Make sure they have your personal email;
  - Take notes of what happened;
  - Get involved with a lawyer before signing anything (e.g. non-compete agreements).
Use LinkedIn:
- Send a note when you connect and keep in touch with past colleagues.
- Ask recruiters about the job market, salary ranges for people with your experience and good skills to learn.
- Review your connections’ companies, review posted jobs and ask for introductions from your connections (let them know why the company is awesome, why you’re awesome and why you want to work there).
Contracting: try it during the first decade of your career.
- It’s easier than applying for a full-time job given you’re easier to let go.
- It will keep your skills sharp if you want to find clients.
- If/when you’re an employee again, job hunting won’t feel as scary.
- You will strengthen your network by being able to refer former colleagues.
- Contracting can be more lucrative on a per-hour basis.
- Contracting via an agency is easier (you avoid sales and accounting) but it pays less.
- Direct contracting is harder but you’ll learn beyond software development (sales, customer support, marketing, invoicing and chasing payments). It leads to more control over the clients you work with and more income. You may spend as much as 50% of your time on non-billable work such as seeking new opportunities: your rate needs to reflect this.
Engineering management: if you move to a management position, be prepared to be a novice again as the skill set involved is not the same.
- You should understand the code but keep out of the critical path.
- You shouldn’t make technical decisions.
- You are here to enable teams to do great work.
- The fun, technical stuff should only be done during hackathons.
- Be clear about what you want and maintain the right set of skills so you can keep doing what you enjoy the most.
Someday, you won’t want to code for a living: becoming a manager gives you leverage to have more impact in the world.
- Impact can be achieved through writing, project management, product management, speaking, starting a business, mentoring, leading a team, managing, teaching, architecting and consulting. Notice that coding is not the primary output.

Community

Building a community/network puts you in a situation where you can help people help others.
Meetups: find one on meetup.com.
- It will expose you to new ideas you can bring to your job.
- You can have professional conversations with low stakes.
- It gives you a way to practice talking to new people and make friends.
- It provides a network outside of your coworkers should you need to find a new job.
Conversational hooks: give some details about where you work and what projects you have been working on to give the chance to the other person to steer the conversation in the direction they want. Flip the question to ask people to talk about themselves (most people like to do that).
- Don’t make transactional conversations: find something interesting to talk about.
Online tech communities:
- Sign up for an email list related to the technologies you are interested in. Sign up for notifications on projects. You can follow project contributors on Twitter or other social media places.
- For something more general, have a look on Slack, forums, IRC channels, Facebook groups and so on. Slashdot, Hacker News, Reddit, Stack Overflow are other good options.
You get what you give:
- Offering to help makes you more empathetic and aware of other people’s needs.
- It feels good to help.
- You become less arrogant.
- You help build the positivity of the community.
- People will be quicker to recommend you for a job.
- It improves your sense of self-worth and confidence.
Build your work community:
- Use LinkedIn.
- Never leave a job on bad terms. Give the requisite notice, document your work and prepare for a hand off.
- Keep in touch with past coworkers, even if it’s just to leave a note on their work anniversary.
Three daily mantras:
- Surround yourself with people smarter than you.
- Build community and give without expecting anything in return.
- Listen to your gut, without exception.
Build a personal board of advisors.
- Some issues you can discuss with them that aren’t topics for current coworkers:
  - Someone under your supervision is not succeeding;
  - You have an issue with your manager that you’re not sure how to solve;
  - You are evaluating job offers.
- Make sure to keep these relationships fresh and offer to help too.
- This circle shifts over time: a new developer doesn’t have the same questions that a team lead would have.

Conclusion

That’s it: we went through a short overview of a book I’ve enjoyed and recommend heartily. You can find more on the author’s website. You might also enjoy a follow-up book summary about The Effective Engineer. May you live this knowledge!

sglavoie.com – Category: Learnings

Reading notes: A Philosophy of Software Design, 2nd Edition

Nature of complexity

Strategic vs. tactical programming

Modules should be deep

Information hiding

General-Purpose Modules are Deeper

Different Layer, Different Abstraction

Pull Complexity Downwards

Better Together Or Better Apart?

Define Errors Out Of Existence

Design it Twice

Why Write Comments? The Four Excuses

Comments Should Describe Things that Aren’t Obvious from the Code

Choosing names

Use comments as part of the design process

Modifying existing code

Consistency

Code should be obvious

Software trends

Designing for performance

Decide what matters

Conclusion

Book summary: 100 Go Mistakes and How to Avoid Them

Go: Simple to learn but hard to master

Go outline

Simple doesn’t mean easy

100 Go mistakes

Code and project organization

1 - Unintended variable shadowing

2 - Unnecessary nested code

3 - Misusing init functions

4 - Overusing getters and setters

5 - Interface pollution

6 - Interface on the producer side

7 - Returning interfaces

8 - any says nothing

9 - Being confused about when to use generics

10 - Not being aware of the possible problems with type embedding

11 - Not using the functional options pattern

12 - Project misorganization

13 - Creating utility packages

14 - Ignoring package name collisions

15 - Missing code documentation

16 - Not using linters

Data types

17 - Creating confusion with octal literals

18 - Neglecting integer overflows

19 - Not understanding floating points

20 - Not understanding slice length and capacity

21 - Inefficient slice initialization

22 - Being confused about nil vs. empty slices

23 - Not properly checking if a slice is empty

24 - Not making slice copies correctly

25 - Unexpected side effects using slice append

26 - Slices and memory leaks

27 - Inefficient map initialization

28 - Maps and memory leaks

29 - Comparing values incorrectly

Control structures

30 - Ignoring the fact that elements are copied in range loops

31 - Ignoring how arguments are evaluated in range loops

32 - Ignoring the impact of using pointer elements in range loops

33 - Making wrong assumptions during map iterations

34 - Ignoring how the break statement works

35 - Using defer inside a loop

Strings

36 - Not understanding the concept of a rune

37 - Inaccurate string iteration

38 - Misusing trim functions

39 - Under-optimized string concatenation

40 - Useless string conversions

41 - Substrings and memory leaks

Functions and methods

42 - Not knowing which type of receiver to use

43 - Never using named result parameters

44 - Unintended side effects with named result parameters

45 - Returning a nil receiver

46 - Using a filename as a function input

47 - Ignoring how defer arguments and receivers are evaluated

8 - `any` says nothing

25 - Unexpected side effects using slice `append`

35 - Using `defer` inside a loop

71 - Misusing `sync.WaitGroup`

72 - Forgetting about `sync.Cond`

73 - Not using `errgroup`

76 - `time.After` and memory leaks

83 - Not enabling the `-race` flag