The Complete Go (Golang) Guide
From zero to production — every concept, with code. After this, you jump straight into Gin.
01 Why Go?
- vs C/C++: Go has garbage collection — no manual
malloc/free, no use-after-free, no double-free. No pointer arithmetic. No header files. Compiles in seconds, not minutes. - vs Java/C#: No JVM/CLR — Go compiles to a static native binary. No reflection-heavy frameworks. Startup is <10ms instead of multi-second JVM warmup. Memory footprint is ~10× smaller.
- vs Python/Node.js: Statically typed (catches bugs at compile time), 10–100× faster on CPU work, true parallelism via goroutines (not just async I/O on a single thread).
- vs Rust: Go gives up some peak performance and memory safety guarantees for radical simplicity — no lifetimes, no borrow checker, no traits/generics gymnastics. Faster to learn, faster to ship.
- vs Ruby/PHP: Compiled, not interpreted. Concurrent by default. No "global interpreter lock" issues.
scp ./myapp server: — no Python virtualenvs, no node_modules, no JVM tuning. Goroutines let one process handle millions of concurrent connections with a few KB of stack each. The strict opinionated style (one formatter, one test runner, one way to do most things) means any Go codebase feels familiar within minutes.Go was created at Google in 2007 by Robert Griesemer, Rob Pike, and Ken Thompson. It was designed to solve real problems:
- Fast compilation — compiles to native machine code in seconds
- Built-in concurrency — goroutines and channels are first-class citizens
- Garbage collected — no manual memory management, but still fast
- Static typing — catches bugs at compile time, not runtime
- Simple syntax — only 25 keywords (compare: C++ has 95+)
- Single binary — compiles to one binary, no dependencies to install
- Standard library — HTTP server, JSON, crypto, testing — all built-in
break case chan const continue default defer else fallthrough for func go goto if import interface map package range return select struct switch type var
Go is used by: Docker, Kubernetes, Terraform, Prometheus, CockroachDB, Hugo, Caddy, Uber, Twitch, Dropbox, Cloudflare.
- Cloud infrastructure & DevOps tools — Docker, Kubernetes, Terraform, Helm — single binary deployment is unbeatable
- High-performance API backends — Uber dispatch, Cloudflare edge, Twitch chat — handles millions of concurrent connections
- CLI tools — gh, hugo, fzf, Vault — compiles to a static binary you just drop on any machine
- Microservices — fast startup (no JVM warmup), low memory footprint, perfect for Kubernetes pods
- Network proxies & load balancers — Caddy, Traefik — goroutines map naturally to per-connection concurrency
- Distributed databases — CockroachDB, InfluxDB, etcd — concurrency primitives + GC make this manageable
02 Setup & Hello World
go. This one binary handles building (go build), running (go run), testing (go test), formatting (go fmt), vetting (go vet), dependency management (go mod), documentation (go doc), and even profiling (go tool pprof). A "Hello, World" Go program is just 5 lines: a package declaration, an import, and a main() function that prints.- vs Node.js: Node needs
node+npm+tsc+eslint+prettier+jest+nodemonas separate tools, each with its own config file. Go has all of this built into one binary. - vs Python: Python needs
python+pip+venv/poetry+black+pytest+mypy. Go has none of these — no virtualenv ever needed because dependencies are versioned per-module ingo.mod. - vs Java: No
JAVA_HOME, noCLASSPATH, no Maven/Gradle, nopom.xml. Justgo build. - vs C/C++: No
Makefile, noCMakeLists.txt, noautotools. Builds are reproducible across machines because the toolchain is opinionated about layout.
go build immediately. The same toolchain you use locally runs in CI with no extra setup, eliminating "works on my machine" almost entirely. Cross-compilation is a first-class feature: GOOS=linux GOARCH=arm64 go build produces a Linux ARM binary from your Mac, no Docker required.- Unused imports = compile error. Not a warning, an error.
- Unused local variables = compile error. (Unused package-level vars are allowed.)
- Opening brace MUST be on the same line as
func,if,for— never on the next line. - One package per directory. All
.gofiles in a folder must declare the same package name. - No semicolons. The lexer inserts them automatically at line ends.
golang:1.22-alpine, final stage is FROM scratch + the static binary — typical image is <20 MB. In CI, just go build ./... + go test ./... — no installation steps. For monorepos, put each binary under cmd/<name>/main.go sharing private code from internal/.Installation
# macOS
brew install go # installs Go via Homebrew
# Linux
wget https://go.dev/dl/go1.22.0.linux-amd64.tar.gz # download Go tarball
sudo tar -C /usr/local -xzf go1.22.0.linux-amd64.tar.gz # extract to /usr/local
export PATH=$PATH:/usr/local/go/bin # add go binary to PATH
# Verify
go version # go version go1.22.0 darwin/arm64
Your First Program
package main // every file must declare its package
import "fmt" // import the fmt (format) package
func main() { // program entry point
fmt.Println("Hello, World!") // print with newline
}
# Run it
go run main.go # compile + run in one step
# Build a binary
go build -o myapp main.go # -o names the output binary
./myapp # execute the compiled binary
- Every Go file starts with
packagedeclaration - The
mainpackage withfunc main()is the entry point - Unused imports = compile error (Go is strict!)
- Unused variables = compile error
- Opening brace
{MUST be on the same line (not next line) - No semicolons needed (compiler inserts them)
- Docker images — multi-stage builds: build stage compiles, final stage is
FROM scratch+ the binary (often <20MB) - CI/CD —
go buildin GitHub Actions / GitLab CI produces artifacts with no runtime install on the target - Cross-compilation —
GOOS=linux GOARCH=arm64 go buildfrom a Mac to deploy to Linux ARM servers - Monorepo — multiple
cmd/*/main.goentry points shareinternal/packages
03 Modules & Packages
go.mod file at its root. A package is a single directory of .go files sharing a namespace. Modules are Go's unit of dependency management and versioning; packages are its unit of code organization and visibility. The go.mod file declares the module path (typically the repo URL), the minimum Go version, and the list of required dependencies. A go.sum file alongside it pins cryptographic checksums of every dependency for reproducible, tamper-proof builds.- vs npm: One npm package = one repo with one entry point. One Go module can contain hundreds of packages, each independently importable. The module path doubles as the import URL — there's no central registry like npmjs.com.
- vs Python: Python imports are file-based; Go imports are directory-based. Every
.gofile in a folder must declare the same package name. No__init__.pyrequired. - vs Java: No Maven Central, no group/artifact IDs. The import path is the URL where the source lives. Visibility uses case, not
public/privatekeywords. - vs Rust: Cargo is similar in spirit but uses a registry (crates.io). Go is fully decentralized — any git repo can be a module.
go.sum checksums, semantic versioning via git tags (v1.2.3), and a globally unique namespace tied to your repo URL — collisions are mathematically impossible. The internal/ convention is enforced by the compiler: any directory under internal/ can only be imported by code within the parent module, giving you real encapsulation across module boundaries. The cmd/ + internal/ + pkg/ layout lets one module produce multiple binaries that share private business logic.- Uppercase first letter (
User,GetName,MaxRetries) = exported, importable from outside the package. - Lowercase first letter (
user,getName,maxRetries) = unexported, only visible within the same package. - This applies to everything: types, functions, variables, constants, struct fields, methods, even individual struct tags.
go mod init github.com/you/project — start a new module. go mod tidy — sync go.mod with imports actually used. go get pkg@v1.2.3 — add or update a dependency. go mod vendor — copy deps into a vendor/ folder for air-gapped builds. The replace directive in go.mod lets you redirect a module to a local path during development across multiple repos.Creating a Module
# Initialize a new module
mkdir myproject && cd myproject # create and enter project dir
go mod init github.com/yourname/myproject # creates go.mod with module path
# This creates go.mod file
go.mod file
module github.com/yourname/myproject // unique module path
go 1.22 // minimum Go version required
require ( // external dependencies
github.com/gin-gonic/gin v1.9.1 // web framework
github.com/go-sql-driver/mysql v1.7.1 // MySQL driver
)
Package System — File Structure
- One folder = one package (all
.gofiles in a folder must have the same package name) - Exported = starts with Uppercase (e.g.,
User,GetName()) - Unexported = starts with lowercase (e.g.,
user,getName()) — private to package internal/packages can only be imported by code within the parent directory
Using Packages
// pkg/models/user.go
package models // folder name = package name
type User struct { // Uppercase = exported type
ID int // Uppercase = exported field
Name string // accessible outside package
age int // lowercase = unexported, private to this package
}
// Exported function — accessible from outside
func NewUser(id int, name string, age int) User {
return User{ID: id, Name: name, age: age} // set unexported field here
}
// unexported function — only accessible within 'models' package
func validate(u User) bool {
return u.age > 0 // age accessible inside package
}
// main.go
package main // entry point package
import ( // grouped imports
"fmt" // standard library
"github.com/yourname/myproject/pkg/models" // our package
)
func main() {
u := models.NewUser(1, "Yatin", 25) // create user via constructor
fmt.Println(u.Name) // OK: Name is exported
// fmt.Println(u.age) // ERROR: age is unexported
}
internal/for business logic —internal/auth,internal/billingcan't be imported by other modules even if your repo is publiccmd/api+cmd/worker— same module, two binaries: a web server and a background job runner sharinginternal/db- Replace directive —
replace github.com/x/y => ../yin go.mod for local development across two repos - Private modules —
GOPRIVATE=github.com/yourorg/*bypasses the public proxy for private repos - Versioning — git tag
v1.2.3on your repo and any consumer cango get yourrepo@v1.2.3
04 Variables & Types
var name type = value (works anywhere, including package level) and the short form name := value (only inside functions; the type is inferred from the initializer). Every variable has a zero value if not explicitly initialized: 0 for numerics, "" for strings, false for bools, nil for pointers/slices/maps/channels/functions/interfaces.- Numeric:
int,int8,int16,int32,int64,uint,uint8(=byte),uint16,uint32,uint64,float32,float64,complex64,complex128,uintptr. - Text:
string(immutable bytes),rune(=int32, one Unicode code point). - Other:
bool,error(interface). - Composite: arrays (
[N]T), slices ([]T), maps (map[K]V), channels (chan T), structs, interfaces, function types, pointers (*T).
- vs JavaScript/Python: Types are fixed and checked at compile time. There is no runtime
TypeErrorfrom passing a string where a number was expected — the code simply won't compile. - vs C: Integer sizes are defined (
int32is always 32 bits, never platform-dependent). There are no implicit numeric conversions —int + int64is a compile error; you must writeint64(x) + y. - vs Java: The short form
x := 10means you rarely write the type explicitly. No autoboxing — primitive numerics never become "objects". - vs TypeScript: Go's types are real (not erased). The same type system runs at compile time and is reflected at runtime via the
reflectpackage.
type UserID int64 and type OrderID int64 are both int64 underneath but you cannot pass one where the other is expected. This eliminates an entire class of bugs (mixing up similar-looking IDs) at compile time. Standard library examples: time.Duration, http.StatusCode, os.FileMode.:=can shadow outer variables if you're not careful — use=to reassign.- The default
intis platform-dependent (32 or 64 bits) — use explicit sizes likeint64when interop matters. - Floating-point comparisons need an epsilon:
math.Abs(a-b) < 1e-9, nevera == b. nilis typed: anil*intis not interchangeable with anil*User.
Declaring Variables
package main // declares this file belongs to main
import "fmt" // needed for Println
func main() {
// Method 1: var keyword (can be used anywhere)
var name string = "Yatin" // explicit type + value
var age int = 25 // explicit int type
// Method 2: type inference
var city = "Delhi" // Go infers string
// Method 3: short declaration := (ONLY inside functions)
country := "India" // most common style
pi := 3.14 // inferred as float64
isActive := true // inferred as bool
// Method 4: multiple declarations
var (
x int = 10 // grouped vars are easier to read
y float64 = 20.5
z string = "hello"
)
// Multiple short declarations
a, b, c := 1, "two", 3.0 // each gets its own type
fmt.Println(name, age, city, country, pi, isActive, x, y, z, a, b, c) // use all vars
}
Zero Values (defaults when not initialized)
| Type | Zero Value | Example |
|---|---|---|
int, int8, int16, int32, int64 | 0 | var x int → 0 |
uint, uint8...uint64 | 0 | var x uint → 0 |
float32, float64 | 0.0 | var x float64 → 0 |
bool | false | var x bool → false |
string | "" | var x string → "" |
pointer, slice, map, channel, func, interface | nil | var x []int → nil |
Type Conversions (Go has NO implicit conversions)
func main() {
var i int = 42 // start with an int
var f float64 = float64(i) // explicit conversion required
var u uint = uint(f) // explicit
// String conversions
s := strconv.Itoa(42) // int → string: "42"
n, _ := strconv.Atoi("42") // string → int: 42
fs := fmt.Sprintf("%.2f", 3.14) // float → string: "3.14"
// byte slice ↔ string
bytes := []byte("hello") // string → []byte
str := string(bytes) // []byte → string
_ = i; _ = f; _ = u; _ = s; _ = n; _ = fs; _ = str // suppress unused-var errors
}
Type Aliases & Custom Types
// Custom type — creates a NEW type
type UserID int64 // prevents mixing IDs with plain ints
type Celsius float64 // named type for type safety
type Fahrenheit float64 // separate type from Celsius
func main() {
var id UserID = 12345 // can't accidentally pass as plain int64
var temp Celsius = 36.6 // body temperature in Celsius
// Cannot mix types even if underlying type is same
// var f Fahrenheit = temp // ERROR!
var f Fahrenheit = Fahrenheit(temp*9/5 + 32) // convert formula
_ = id; _ = f // discard to avoid unused-var error
}
// Type alias — just another name (same type)
type byte = uint8 // this is how byte is defined in Go
type rune = int32 // this is how rune is defined in Go
- Custom ID types —
type UserID int64,type OrderID int64prevent passing a UserID where an OrderID is expected (caught at compile time, not in production) - Domain types —
type Email string,type SafeHTML stringlet methods enforce validation/escaping - Money —
type Cents int64avoids floating-point rounding errors in billing systems - Type-safe units —
time.Durationis justtype Duration int64; using a named type prevents passing seconds where milliseconds are expected :=vsvar— use:=inside functions,varat package level (where:=isn't allowed)
05 Constants & Iota
const keyword. Constants live in the binary itself — they're inlined wherever used and incur zero runtime overhead. They come in two flavors: typed constants (e.g. const x int32 = 42) which behave like normal typed values, and untyped constants (e.g. const Pi = 3.14159) which have arbitrary precision and only acquire a concrete type at the point of use. The special identifier iota is a constant generator that auto-increments inside each const ( ... ) block, starting at 0 — the standard idiom for enums and bit flags.const Pi = 3.141592653589793238462643383279502884 can be used as float32, float64, or even complex128 depending on context, without losing precision until conversion. They participate in expressions with other constants without type promotion: const Big = 1 << 100 compiles fine even though it overflows any concrete integer type — as long as you don't assign the whole thing to one.- Sequential enum:
const ( Sunday = iota; Monday; Tuesday; ... )→ 0, 1, 2, ... - Bit flags:
const ( Read = 1 << iota; Write; Execute )→ 1, 2, 4 (powers of two for bitwise OR). - Skip values: Use
_to skip —const ( _ = iota; KB = 1 << (10 * iota); MB; GB; TB ). - Custom typed enums:
type Status int; const ( Active Status = iota; Inactive; Banned ).
- vs JS
const: JavaScript'sconstonly freezes the binding — the underlying object can still mutate. Go constants are truly immutable and can only hold primitive values, not arrays or maps. - vs Java
final: Javafinalcan be assigned at runtime (final long now = System.currentTimeMillis()); Go constants must be computable at compile time.const x = time.Now()will not compile. - vs C
#define: Constants are typed, scoped, and visible to the debugger — none of the preprocessor footguns. - vs Rust
const: Similar in spirit but Rust has bothconstandstatic; Go has onlyconstfor compile-time and package-levelvarfor runtime singletons.
iota turns "implement an enum" from a 30-line dance (Java) into 3 lines. They're the cleanest way to express state machines, status codes, log levels, file permissions, and any other fixed set of values.const slice, map, struct, or function. For "constant-ish" composite values, use a package-level var with a comment, or a function that returns a fresh copy.const Pi = 3.14159 // untyped constant, very precise
const AppName = "MyApp" // string constant, immutable
// Group of constants
const (
StatusOK = 200 // HTTP 200 OK
StatusNotFound = 404 // HTTP 404 Not Found
StatusError = 500 // HTTP 500 Server Error
)
// iota — auto-incrementing constant generator (starts at 0)
type Weekday int // custom int type for weekdays
const (
Sunday Weekday = iota // 0
Monday // 1
Tuesday // 2
Wednesday // 3
Thursday // 4
Friday // 5
Saturday // 6
)
// iota with bit shifting — permission flags
type Permission uint // bitmask type for permissions
const (
Read Permission = 1 << iota // 1 (001)
Write // 2 (010)
Execute // 4 (100)
)
func main() {
myPerms := Read | Write // 3 (011)
if myPerms&Read != 0 { // bitwise AND to check flag
fmt.Println("Has read permission")
}
}
- Enum-like states —
type OrderStatus intwithiotafor Pending/Paid/Shipped/Delivered/Cancelled - Permission bitmasks — file permissions, RBAC roles, feature flags packed into a single int
- HTTP status constants — already in stdlib (
http.StatusOK, etc.) — never hardcode 200/404/500 - Untyped constants —
const Pi = 3.14159works with bothfloat32andfloat64contexts without conversion - Compile-time config —
const DebugMode = false— dead-code elimination removes all debug branches at build time
06 Operators
int + int64 won't compile.- Arithmetic:
+,-,*,/,%(modulo, integer types only). - Comparison:
==,!=,<,>,<=,>=. All returnbool. - Logical:
&&,||,!. Short-circuit evaluation, just like C. - Bitwise:
&(AND),|(OR),^(XOR — also unary NOT),<<(left shift),>>(right shift),&^(AND NOT — bit clear, unique to Go). - Assignment:
=,:=(declare + assign), and compound forms+=,-=,*=,/=,%=,&=,|=,^=,<<=,>>=,&^=. - Misc:
&(address-of),*(dereference),<-(channel send/receive),...(variadic spread).
- No ternary operator —
cond ? a : bdoesn't exist. You must write a fullifstatement. The Go team explicitly rejected ternary because it produces hard-to-read nested chains. ++and--are statements, not expressions.x++is valid as a standalone line, but you can never writey = x++orarr[i++]. Also no prefix form++x.- No implicit conversions.
int(x) + int(y)won't compile if one isint32and the other isint64— you must explicitly convert. - No operator overloading. Custom types cannot define
+or==.big.Int.Add(a, b)is method-based. - Simpler precedence than C — only 5 levels instead of 15. This eliminates whole categories of "wait, does
&bind tighter than+?" confusion. - Bit-clear
&^is a unique convenience:x &^ yis the same asx & (^y), useful for clearing bits in a flag set.
int to an int64. The lack of ++ as an expression eliminates the C/Java "what does a[i++]=i++ do?" undefined-behavior trap entirely.7 / 2 == 3, not 3.5. Use float64(a)/float64(b) for real division. Modulo of negatives follows the dividend's sign: -7 % 3 == -1 in Go (different from Python's 2). Unsigned underflow wraps silently: var x uint = 0; x-- gives the maximum uint, no panic.// Arithmetic: + - * / %
// Comparison: == != < > <= >=
// Logical: && || !
// Bitwise: & | ^ << >> &^
// Assignment: = := += -= *= /= %= &= |= ^= <<= >>=
// Address: & (address of) * (dereference)
// Channel: <- (send/receive)
func main() {
// Go has NO ternary operator (no ? :) — must use if-else
x := 10 // value to test
var result string // declared before if-else scope
if x > 5 {
result = "big" // assign inside if-branch
} else {
result = "small" // assign inside else-branch
}
// Bit clear (AND NOT) — unique to Go
a := 0b1010 // binary literal: 10
b := 0b1100 // binary literal: 12
fmt.Printf("%04b\n", a&^b) // 0010
_ = result // discard to avoid unused-var error
}
- Bitwise OR for flags —
flags := READ | WRITE | EXECpacks multiple booleans into one int (used in syscalls, file open modes) - Bitwise AND for testing flags —
if perms & READ != 0— standard pattern inos.OpenFile, network protocol parsing - Modulo for sharding —
shardID := userID % numShardsdistributes data across DB shards - Bit clear (
&^) — useful for unsetting flags:flags = flags &^ EXECremoves the EXEC bit - Pointer ops (
&,*) — passing large structs to functions without copying, mutating receiver state in methods
07 Control Flow
if, for, switch) plus the helpers break, continue, return, goto, and the rare fallthrough. There is exactly one looping keyword — for — which subsumes while, do-while, and C-style for loops via different syntactic forms.- Standard
if:if cond { ... } else if ... { ... } else { ... }— no parentheses, mandatory braces. ifwith init:if v, err := f(); err != nil { ... }— declaresvanderrscoped to theif/elseblocks only.- C-style for:
for i := 0; i < 10; i++ { ... } - While-style for:
for cond { ... } - Infinite loop:
for { ... }(withbreakto exit) - Range loop:
for i, v := range slice { ... }— works on slices, arrays, maps, strings, and channels. - Value
switch:switch x { case 1: ... case 2, 3: ... default: ... } - Tagless
switch:switch { case x > 0: ... case x < 0: ... }— a clean if-else chain. - Type switch:
switch v := x.(type) { case int: ... case string: ... }
- No
whileordo-while— Go realized you don't need three loop keywords when one suffices. Usefor cond { }as a while loop. - Conditions take no parentheses, but braces are mandatory — even single-line bodies need
{ }. This eliminates the C/Java dangling-else and Apple-goto-fail bug class. - Switch cases don't fall through by default — the exact opposite of C/Java/JS. You must explicitly write
fallthroughon the rare occasion you want it. This eliminates an entire category of "I forgot tobreak" bugs. - Switch is much more powerful — it can compare values, ranges (via tagless form), types (via
.(type)), and supports multiple values per case. - The init statement on
ifandswitchis unique to Go among mainstream languages — it scopes temporary variables tightly.
if err := f(); err != nil) is the canonical Go idiom — it keeps error variables scoped to the block they're checked in, preventing accidental reuse below. Switch's no-fallthrough default means you can read a switch top to bottom without having to mentally track which cases have break. The forced braces remove a common bug source.- Slice/array:
for i, v := range s—iis index,vis a copy of the element. - Map:
for k, v := range m— order is randomized on every iteration! - String:
for i, r := range s—iis byte offset,ris arune(decoded UTF-8 code point). - Channel:
for v := range ch— receives until the channel is closed. - Integer (Go 1.22+):
for i := range 10— loops 0..9. - Loop variable scoping (Go 1.22+): Each iteration gets a fresh
iandv, fixing the long-standing closure-capture footgun.
If-Else (with init statement)
// Standard
if x > 10 { // no parentheses needed around condition
fmt.Println("big")
} else if x > 5 { // chain more conditions
fmt.Println("medium")
} else { // catch-all
fmt.Println("small")
}
// With init statement (v is scoped to the if-else block)
if v := compute(); v > 100 { // init runs before condition
fmt.Println("over 100:", v)
}
// Very common pattern in Go:
if err := doSomething(); err != nil { // run & check error at once
return err // early return on error
}
For Loop (the ONLY loop in Go)
// Classic for
for i := 0; i < 10; i++ { // init; condition; post
fmt.Println(i)
}
// While-style
n := 1 // start value
for n < 100 { n *= 2 } // only condition, like while
// Infinite loop
for { // no condition = loops forever
break // use break/return to exit
}
// Range over slice
nums := []int{10, 20, 30} // slice to iterate
for i, v := range nums { fmt.Println(i, v) } // i=index, v=value
// Range — skip index
for _, v := range nums { fmt.Println(v) } // _ discards index
// Range over map
m := map[string]int{"a": 1, "b": 2} // map literal
for k, v := range m { fmt.Println(k, v) } // k=key, v=value
// Range over string (iterates runes, not bytes)
for i, ch := range "Hello" { fmt.Printf("%d: %c\n", i, ch) } // i=byte offset, ch=rune
// Labels with break/continue
outer: // label for the outer loop
for i := 0; i < 5; i++ {
for j := 0; j < 5; j++ {
if i+j == 4 { break outer } // breaks all the way out
}
}
Switch
// No fall-through by default (no break needed!)
switch day { // switch on variable value
case "Mon", "Tue", "Wed", "Thu", "Fri": // multiple values per case
fmt.Println("Weekday")
case "Sat", "Sun":
fmt.Println("Weekend")
default: // runs if no case matched
fmt.Println("Unknown")
}
// Type switch (very useful with interfaces)
switch v := i.(type) { // i.(type) extracts dynamic type
case int: fmt.Println("int", v) // v is int here
case string: fmt.Println("string", v) // v is string here
default: fmt.Printf("unknown: %T\n", v) // %T prints type name
}
// Expressionless switch (cleaner than if-else chains)
switch { // no value — each case is a bool expression
case score >= 90: fmt.Println("A")
case score >= 80: fmt.Println("B")
default: fmt.Println("F")
}
// fallthrough — explicitly fall to next case
switch 1 { // switch on literal value 1
case 1:
fmt.Println("one")
fallthrough // forces execution into case 2
case 2:
fmt.Println("two") // also prints
}
- Guard clauses with init —
if err := db.Ping(); err != nil { return err }— limits err's scope, idiomatic in every Go HTTP handler - Type switch on interfaces — JSON decoders use
switch v := val.(type) { case map[string]any: ... }to handle dynamic types - Expressionless switch — replaces if/elif chains for grading, threshold checks, status mapping
- For-range over channel —
for msg := range ch { handle(msg) }— natural consumer loop until channel closes - Labeled break — exiting nested loops in matrix scans, search algorithms, retry-with-timeout patterns
08 Functions
func name(params) returns { ... }. Functions are first-class values in Go — you can store them in variables, put them in slices/maps/structs, pass them as arguments, and return them from other functions. The signature reads left-to-right like English: func Add(a int, b int) int. Go supports multiple return values, named returns, variadic parameters (...T), and anonymous functions (a.k.a. lambdas) that can capture surrounding variables (closures).- Multiple return values:
func divmod(a, b int) (int, int) { return a/b, a%b } - Named returns:
func split(sum int) (x, y int) { x = sum * 4 / 9; y = sum - x; return }— barereturnreturns the named values. - Variadic:
func sum(nums ...int) int— call assum(1, 2, 3)orsum(slice...). - Multiple assignment from one return:
v, ok := m[key]— the comma-ok idiom. - Blank identifier:
_, err := f()— discard a return value you don't need. - Anonymous closures:
add := func(a, b int) int { return a + b } - Method values and method expressions:
f := user.Savebinds the receiver into a callable.
- Multiple returns are native — no tuples (Python), no out parameters (C#), no destructuring tricks. The compiler treats them as first-class.
- No function overloading — each name maps to exactly one function. Forces meaningful names like
ParseInt/ParseFloatinstead of overloadedparse(...). - No default arguments — use the functional-options pattern instead (
NewServer(WithPort(8080), WithTLS(cert))). - No method overloading either — one method name per type.
- Parameter type goes after the name —
x int, notint x. Multiple params of the same type can share:func add(a, b, c int). - Functions are values — much more so than Java but similar to JS/Python first-class functions.
(value, error) pattern — every fallible call is honest about it in its signature. No overloading means one function name = one behavior, which makes "find usages" and code review trivial. The lack of default args pushes you toward the explicit functional-options pattern, which scales better as APIs grow.- (value, error) return:
data, err := json.Marshal(x)— universal Go shape. - Higher-order middleware:
func Logger(next http.Handler) http.Handler— wrap a handler to add behavior. - Functional options:
NewClient(WithTimeout(5*time.Second), WithRetries(3)). - Defer + named return for error wrapping:
defer func() { err = fmt.Errorf("Foo: %w", err) }() - Closures in
http.HandlerFunc: capture dependencies (db, logger) without globals.
// Basic function
func add(a, b int) int { return a + b } // a,b share type int
// Multiple return values
func divide(a, b float64) (float64, error) { // result + error pattern
if b == 0 { return 0, fmt.Errorf("division by zero") } // guard clause
return a / b, nil // nil means no error
}
// Named return values
func swap(a, b string) (first, second string) { // pre-declared return vars
first = b; second = a // assign named returns directly
return // naked return
}
// Variadic functions
func sum(nums ...int) int { // ... accepts 0 or more ints
total := 0 // accumulator variable
for _, n := range nums { total += n } // add each number
return total
}
sum(1, 2, 3) // pass multiple args directly
numbers := []int{1, 2, 3} // existing slice
sum(numbers...) // spread slice
// Functions are first-class — assigned to variables
double := func(n int) int { return n * 2 } // anonymous func as var
fmt.Println(double(5)) // 10
// Function as parameter (higher-order)
func apply(nums []int, fn func(int) int) []int { // fn is a func param
result := make([]int, len(nums)) // pre-allocate output slice
for i, v := range nums { result[i] = fn(v) } // call fn on each element
return result
}
// init() — runs before main(), used for setup
func init() { // auto-called by runtime at startup
// package-level vars → init() → main()
fmt.Println("initializing...")
}
- (value, error) return — universal Go pattern:
data, err := json.Marshal(x),row, err := db.Query(...) - Variadic functions —
fmt.Printf,append,log.Printfall use them; also options pattern:NewServer(WithPort(8080), WithTLS(cert)) - Higher-order functions — middleware in HTTP routers (
func(http.Handler) http.Handler), retry wrappers, sort comparators - Named returns — useful in functions with deferred error wrapping:
defer func() { err = wrap(err) }() - Closures over loop vars — capture goroutine arguments correctly (Go 1.22+ fixes this automatically)
09 Naming Conventions
HTTPServer, not HttpServer; userID, not userId).- PascalCase: exported types, functions, constants, vars, struct fields, methods. Example:
CalculateArea,UserInfo,MaxRetries. - camelCase: unexported (private) versions of all of the above. Example:
calculateArea,userInfo,maxRetries. - Acronyms: always all caps in both PascalCase and camelCase contexts.
HTTPClient,parseURL,getJSONPayload,userID. NeverHttpClientorparseUrl. - Package names: short, lowercase, single word, no underscores.
http,json,fmt,strconv. Neverhttp_clientorHTTPClient. - File names: snake_case if multi-word:
user_handler.go,tcp_server.go. Test files end in_test.go. - Receivers: short, often 1–2 letters reflecting the type.
func (u *User) Save(),func (s *Server) Start(). Neverselforthis. - Local variables: short and contextual. Loop index =
i, byte buffer =buf, context =ctx, error =err. - Constructors:
NewXreturns a value,NewXFromYfor alternative constructors. Returning*Xis common. - Interface names: often
-ersuffix for single-method interfaces —Reader,Writer,Stringer,Closer. - Errors: exported sentinel errors are
ErrXxx:io.EOF,sql.ErrNoRows. Error types areXxxError:os.PathError.
- vs Java/C#: No
public/private/protectedkeywords — case is the access modifier. Java prefers verbose names likeUserAccountManagerFactory; Go prefers short names likeusers. - vs Python: Python uses
_underscoreas a "private by convention" hint with no enforcement; Go's case rule is compiler-enforced. - vs JavaScript: JS has
#privatefields now but the wider ecosystem is inconsistent. Go is uniform. - vs Ruby: Ruby uses naming for hints (
@instance,@@class,$global) but allows almost anything; Go is much stricter.
HttpClient vs HTTPClient because golint and staticcheck will flag the wrong one. Tying visibility to case removes a whole keyword from the language and lets you instantly tell, while skimming, what's public API and what's internal. Short receiver/variable names reduce visual noise; the convention is "name length should be proportional to scope" — a one-line loop can use i; a package-level config uses databaseURL.gofmt handles whitespace and braces. go vet catches shadowed variables and bad printf verbs. golint (deprecated, succeeded by staticcheck and revive) flags naming-rule violations. Most teams run golangci-lint, which bundles all of these and 50+ more checks.Go has strict and opinionated naming conventions. The case of the first letter determines visibility — this replaces public/private keywords found in other languages.
| Convention | Usage | Example |
|---|---|---|
| PascalCase | Exported (public) — structs, interfaces, functions, vars | CalculateArea, UserInfo, NewHTTPRequest |
| camelCase | Unexported (private) — internal to the package | calculateArea, userID, isValid |
| ALLCAPS | Constants (by convention, not enforced) | MAXRETRIES, GRAVITY |
| Short names | Loop vars, receivers, params with small scope | i, v, r, ctx |
// PascalCase = EXPORTED (visible outside package)
type EmployeeGoogle struct {
FirstName string // exported field
LastName string // exported field
Age int // exported field
}
// camelCase = unexported (private to this package)
var employeeID = 1001 // only this package can access it
// Constants by convention
const MAXRETRIES = 5 // screaming case for constants
const GRAVITY = 9.81
// Acronyms stay ALLCAPS: URL, HTTP, ID, API (not Url, Http, Id)
type HTTPClient struct{} // not HttpClient
var userID int // not userId
Uppercase first letter = exported (accessible from other packages). Lowercase first letter = unexported (package-private). There is no public/private keyword in Go — the case IS the access modifier.
- Receiver names — single letter matching the type:
func (u *User) Save(), neverselforthis - Interface naming —
-ersuffix for single-method interfaces:Reader,Writer,Stringer,Closer - Acronyms stay capitalized —
UserID,HTTPClient,URLParser(golangci-lint enforces this) - Error variables — prefix with
Err:ErrNotFound,ErrTimeout— convention used in all stdlib packages - Test functions —
func TestSomething(t *testing.T)— required for the test runner to find them
10 init() & os.Exit
init() is a special, parameterless, return-less function that the runtime calls automatically before main() for every package that's imported (transitively). Each file can have multiple init() functions and they all run, in source order. os.Exit(code) is the explicit "kill the program now" call: it terminates the process with the given status code and does not run deferred functions. Status code 0 = success, anything else = failure.- Package
main's dependencies are initialized first, in topological order — a package's deps are fully initialized before the package itself. - Within a package, all package-level variable declarations are evaluated first (in dependency order, not source order).
- Then all
init()functions in the package run, file by file, in the order the files appear to the compiler (typically alphabetical). - Finally,
main.main()runs. - If
init()panics, the program never starts.
- vs Java static blocks: Java has
static {}blocks but their order across classes is lazy (on first use). Go's order is eager and fully specified beforemainruns. - vs Python
__init__.py: Python module init runs on first import; Go runs all package inits beforemain. - vs C++ static constructors: Go's order is well-defined and deterministic; C++'s "static initialization order fiasco" is gone.
os.Exitvspanic:panicunwinds the stack, runsdeferfunctions, can be caught withrecover.os.Exitterminates immediately, skipping all of that — use it at the very top ofmainafter error logging.- vs
returnfrom main:returnfrommainexits with status 0 and runs deferred functions;os.Exit(0)exits with status 0 but skips defers.
init() powers the famous "side-effect import" pattern: import _ "github.com/lib/pq" imports the Postgres driver purely for its init(), which registers itself with database/sql. The same trick is used by net/http/pprof, image format decoders (image/png, image/jpeg), and most plugin-style architectures. os.Exit with non-zero codes is essential for CLI tools that need to integrate with shell scripts (set -e) and CI pipelines, which check the exit code to mark a step as failed.- Don't use
init()for things that can fail in interesting ways — there's no way to handle errors gracefully. A panickinginit()kills the binary at startup with no chance to log nicely. os.Exitbypasses defers, so files won't be flushed and locks won't be released. Reserve it for the last line ofmainafter explicit cleanup.- Multiple
init()s in one file all run, in source order — but relying on this is fragile; prefer oneinit()per file. - Hidden global state in
init()makes packages hard to test in isolation. Use explicitNewX()constructors when you can.
init() Function
The init() function runs automatically before main(). It's used for package-level setup like loading config, checking environment, or validating state. You can have multiple init() functions in the same file — they run in order of declaration.
package main
import "fmt"
// You can have MULTIPLE init() functions in one file!
func init() { fmt.Println("Initializing package1...") } // runs 1st
func init() { fmt.Println("Initializing package2...") } // runs 2nd
func init() { fmt.Println("Initializing package3...") } // runs 3rd
func main() {
fmt.Println("Inside the main function")
}
// Output:
// Initializing package1...
// Initializing package2...
// Initializing package3...
// Inside the main function
// Execution order: package-level vars → init() → main()
os.Exit()
os.Exit() terminates the program immediately with a status code. It does NOT run deferred functions — this is a key difference from a normal return.
import (
"fmt"
"os"
)
func main() {
defer fmt.Println("Deferred statement") // will NOT run!
fmt.Println("Starting the main function")
os.Exit(1) // exit code 1 = error, 0 = success
fmt.Println("End of main function") // never reached
}
// Output:
// Starting the main function
// (program exits — no deferred output!)
Unlike return, os.Exit() kills the process instantly. Deferred functions, cleanup code, and file flushes will NOT execute. Use it only for fatal errors or CLI tools where you need a specific exit code.
- Database driver registration —
init()indatabase/sql/driverpackages:sql.Register("postgres", &Driver{}) - Codec/format registration — image format decoders (
image/png,image/jpeg) register themselves viainit() - Flag parsing — Prometheus client libs use
init()to register metrics with the global registry - os.Exit codes — CLI tools follow Unix convention: 0=success, 1=general error, 2=usage error, 130=interrupted (used by gh, kubectl)
- log.Fatal vs os.Exit —
log.Fatalwrites the message then callsos.Exit(1)— defers still don't run - Graceful shutdown — never use
os.Exitin servers; instead listen for SIGTERM and let defers/cleanups run
11 Arrays & Slices
[5]int and [10]int are different, incompatible types. Arrays are value types — assigning or passing one copies every element. A slice is a dynamic, growable view backed by an underlying array. Internally, a slice is a tiny struct of three words: {pointer, length, capacity}. Slices are the everyday "list" type in Go; raw arrays are rare in idiomatic code.- Pointer — to the first element of the backing array.
- Length (
len(s)) — number of elements you can access. - Capacity (
cap(s)) — total space in the backing array from the slice's start.
append would exceed capacity, Go allocates a new, larger backing array (typically 2× or 1.25× depending on size), copies the old elements over, and returns a slice pointing to the new array. This is why s = append(s, x) is the canonical pattern — the returned slice may be different from the input.
- Literal:
s := []int{1, 2, 3} - make with length:
s := make([]int, 5)→ length 5, all zeros, capacity 5. - make with length + capacity:
s := make([]int, 0, 100)→ length 0, capacity 100. The pre-allocation prevents repeated re-grows. - Slicing an existing slice/array:
s[2:5],s[:5],s[2:],s[:]— all O(1), share the backing array. - Three-index slice:
s[2:5:7]— limits the new slice's capacity to 5 (7-2), preventing it from extending into the original.
- vs Java/C# arrays: Go arrays are value types — passing one to a function copies it entirely. Java/C# arrays are reference types.
- vs Python lists / JS arrays: Go slices are strongly typed (
[]intonly holds ints) and contiguous in memory — much faster cache locality, no boxing. - vs C arrays: Slices know their length — there's no separate
lengthparameter, no array-decay-to-pointer, and no buffer overflows pastlen. - vs Rust
Vec: Go slices are simpler (no ownership), but with the same shared-backing-array gotchas you don't get in Rust.
append can silently mutate a slice you passed to a function — or not, depending on capacity.- Append may or may not mutate the original. After
b := append(a, x),amay or may not seexdepending on whethercap(a) > len(a). Always reassign:a = append(a, x). - Subslices share memory.
b := a[2:5]means writes throughbare visible throughaand vice versa. - Slice keeps the backing array alive. Slicing a 1 GB file's bytes to extract 10 chars keeps the whole GB in memory. Use
copyto detach. - Pre-allocate when you know the size:
make([]T, 0, n)avoids repeated re-grows in tight loops. - Nil slice vs empty slice: both have length 0; both work with
append; the difference matters only for JSON marshaling and pointer equality.
Arrays (fixed size — rarely used directly)
var a [5]int // [0 0 0 0 0]
b := [3]string{"a", "b", "c"} // array literal with size
c := [...]int{1, 2, 3} // compiler counts: [3]int
// [3]int and [5]int are DIFFERENT types
// Arrays are VALUE types — assignment copies entire array
x := [3]int{1, 2, 3} // original array
y := x // full copy
y[0] = 99 // modify the copy only
fmt.Println(x) // [1 2 3] — unchanged!
_ = a; _ = b; _ = c // suppress unused-var errors
Slices (dynamic — 99% of the time)
// A slice has: pointer, length, capacity
s1 := []int{1, 2, 3} // literal
s2 := make([]int, 5) // len=5, cap=5, zeros
s3 := make([]int, 3, 10) // len=3, cap=10
var s4 []int // nil slice
// Append
s1 = append(s1, 4, 5) // [1 2 3 4 5]
s1 = append(s1, s2...) // append another slice
// Slicing (shares same underlying array!)
nums := []int{0, 1, 2, 3, 4, 5} // base slice
a := nums[1:4] // [1 2 3]
b := nums[:3] // [0 1 2]
c := nums[3:] // [3 4 5]
// True copy:
copySlice := make([]int, len(nums)) // allocate new backing array
copy(copySlice, nums) // copy src into dst
// Delete element at index i
i := 2 // index to remove
nums = append(nums[:i], nums[i+1:]...) // stitch around index
// Sort
sort.Ints(nums) // sort ints in-place ascending
sort.Slice(people, func(i, j int) bool { // custom sort via less-fn
return people[i].Age < people[j].Age // sort by Age ascending
})
_ = s3; _ = s4; _ = a; _ = b; _ = c // suppress unused-var errors
A slice header is: { ptr *Array, len int, cap int }
When append exceeds capacity, Go allocates a new array (2x for small, 1.25x for large) and copies elements. Always use: s = append(s, x)
- HTTP request bodies —
body, _ := io.ReadAll(r.Body)returns a[]byteslice - DB query results — collecting rows into a
[]Userfor serialization - Pre-allocate with cap —
users := make([]User, 0, len(rows))avoids re-allocation in hot paths - Pagination —
page := items[offset:offset+limit]— slicing creates a view, not a copy - Beware shared backing arrays —
sub := s[1:3]still references original; modifications leak both ways - Arrays for fixed sizes — IPv4 (
[4]byte), MD5 hashes ([16]byte), SHA-256 ([32]byte)
12 Maps
map[K]V, where K is the key type and V is the value type. Keys must be comparable with == — that means anything except slices, maps, and functions (and structs/arrays containing those). Lookups return a special two-value form: v, ok := m[k] — v is the value (or zero value if absent) and ok is a bool telling you whether the key actually existed.- Literal:
m := map[string]int{"a": 1, "b": 2} - Empty with make:
m := make(map[string]int)or with size hintmake(map[string]int, 100) - Insert/update:
m["key"] = 42 - Read:
v := m["key"](returns zero value if absent), orv, ok := m["key"](with existence check). - Delete:
delete(m, "key")— no-op if missing, never panics. - Length:
len(m) - Iterate:
for k, v := range m { ... }— order is randomized! - Clear (Go 1.21+):
clear(m)removes all entries.
- vs Python dict: Iteration order is intentionally randomized on every program run, not insertion-order like Python 3.7+. This is a deliberate language design choice to prevent you from accidentally depending on order.
- vs Java HashMap: Maps are a built-in language type with literal syntax — no
HashMap<String, Integer>ceremony, no autoboxing. - vs C++
std::map: Go's map is a hash table (O(1)), not a tree (O(log n)). Closer tostd::unordered_map. - vs JavaScript objects: Real strong typing on keys and values; no prototype chain weirdness.
- NOT thread-safe: Concurrent reads are fine, but any concurrent write with another read or write triggers a runtime panic. Use
sync.Mapor wrap withsync.RWMutex. - Nil maps: A
nilmap can be read from (returns zero values) and ranged over (zero iterations) but panics on write.
make(map[K]V, n) avoids costly re-hashing as the map grows.- Set:
set := map[string]struct{}{}—struct{}uses zero bytes per value. - Counter:
counts := map[string]int{}; counts[word]++(zero value of int is 0). - Lookup table:
statusText := map[int]string{200: "OK", 404: "Not Found"} - Group-by:
groups := map[string][]Item{}; groups[k] = append(groups[k], item) - Concurrent cache: wrap a regular map with
sync.RWMutex; reach forsync.Maponly for read-mostly workloads with disjoint keys.
m1 := map[string]int{"apple": 5, "banana": 3} // map literal
m2 := make(map[string]int) // empty, ready to use
var m3 map[string]int // nil map — PANICS on write!
// CRUD
m1["cherry"] = 7 // create/update
val := m1["apple"] // read
delete(m1, "banana") // delete
// Check if key exists (comma ok idiom)
if v, ok := m1["grape"]; ok { // ok is false if key missing
fmt.Println("found:", v)
}
// Iterate (order NOT guaranteed)
for k, v := range m1 { fmt.Println(k, v) } // random order each run
// Map of slices
graph := map[string][]string{ // adjacency list pattern
"a": {"b", "c"}, // a connects to b and c
"b": {"d"}, // b connects to d
}
// Maps are NOT safe for concurrent use! Use sync.Map or mutex
_ = m2; _ = m3; _ = val; _ = graph // suppress unused-var errors
- HTTP headers —
http.Headeris amap[string][]stringunder the hood - Caches & lookup tables —
map[string]*Userfor an in-memory user cache (always behind a mutex) - JSON deserialization — unknown shapes go into
map[string]any(formerlyinterface{}) - Counting / frequency —
counts[word]++works because zero value of int is 0 - Set type —
map[string]struct{}uses zero memory for the value, perfect for membership tests - Concurrent access — bare maps panic; use
sync.Mapfor read-heavy orsync.RWMutex+ map otherwise
13 Strings & Runes
string is an immutable, read-only sequence of bytes — typically (but not required to be) valid UTF-8. Internally it's a tiny header of {pointer, length} (16 bytes on a 64-bit machine). A rune is an alias for int32 and represents a single Unicode code point. A byte is an alias for uint8 and represents one raw octet. Indexing a string with s[0] gives you a byte, while ranging with for i, r := range s gives you (byte index, rune) pairs and automatically decodes UTF-8."Go is fire" + emoji has surprises:
len("hello")→ 5 (each ASCII char is 1 byte)len("héllo")→ 6 (theétakes 2 bytes in UTF-8)len("世界")→ 6 (each CJK character takes 3 bytes)len("🔥")→ 4 (the fire emoji is a 4-byte code point)- To count characters, not bytes:
utf8.RuneCountInString(s)orlen([]rune(s)).
- vs Java/C# String: Java strings are sequences of UTF-16 code units (2 bytes each). Go strings are byte sequences with no enforced encoding — though almost always UTF-8.
- vs Python
str: Python 3stris character-oriented and abstracts away encoding. Go is byte-oriented and explicit — sometimes more work, but never surprising. - vs C strings: Not null-terminated! Go strings know their length, so they can contain
\0bytes harmlessly. - vs JS strings: JS uses UTF-16;
"𝄞".length === 2because it's a surrogate pair. Go'slen("𝄞")is 4 (UTF-8 bytes). - Immutable — unlike
[]byte, you cannot dos[0] = 'x'; it's a compile error. - No
chartype — userunefor code points ('A'is a rune literal, value 65) orbytefor raw octets.
s[2:5]) is O(1) with no copy. For mutation-heavy work, convert to []byte or use strings.Builder; for parsing, use strings package functions or bufio.Scanner.- string ↔ []byte:
b := []byte(s)ands := string(b)— both copy. Useunsafe.String/StringData(Go 1.20+) for zero-copy in hot paths. - string ↔ []rune:
r := []rune(s)decodes UTF-8 into a slice of code points.s := string(r)encodes back. - int → string of one rune:
string(65)→"A". (go vetwarns about this; usestrconv.Itoafor "65".) - int → decimal string:
strconv.Itoa(65)→"65".
// Strings are immutable byte slices
emoji := "Go is 🔥" // emoji is multi-byte (UTF-8)
fmt.Println(len(emoji)) // 10 (bytes!)
fmt.Println(len([]rune(emoji))) // 7 (characters)
// String builder (efficient concatenation)
var sb strings.Builder // avoids repeated allocs
for i := 0; i < 1000; i++ { sb.WriteString("hello ") } // append to buffer
result := sb.String() // convert buffer to string
// strings package essentials
strings.Contains(s, "World") // true
strings.HasPrefix(s, "Hello") // true
strings.HasSuffix(s, "!") // true
strings.ToUpper(s) // "HELLO, WORLD!"
strings.TrimSpace(" hi ") // "hi"
strings.Split("a,b,c", ",") // ["a","b","c"]
strings.Join([]string{"a","b"}, "-") // "a-b"
strings.ReplaceAll(s, "l", "L") // "HeLLo, WorLd!"
strings.Repeat("Go", 3) // "GoGoGo"
// Raw strings (backticks)
raw := `no \n escaping here` // useful for regex, JSON, HTML
// Sprintf for formatting
msg := fmt.Sprintf("Name: %s, Age: %d", "Yatin", 25) // build formatted string
_ = result; _ = raw; _ = msg // suppress unused-var errors
len()returns bytes, not characters — bug in user-input validation if you assume otherwise (emojis, accented chars, CJK)- Iterate with
rangefor runes —for _, r := range sdecodes UTF-8 properly; indexings[i]gives a single byte - Raw strings for regex —
`\d+\s+\w+`avoids double-escaping\\d+ - Raw strings for SQL — multi-line queries with embedded quotes stay readable
- Interning — string headers are
{ptr, len}(16 bytes); passing strings by value is cheap because the underlying bytes are shared and immutable
14 String Functions & Builder
strings standard-library package — dozens of helpers for searching, splitting, joining, replacing, trimming, and case-converting strings. Plus the strings.Builder type, which gives you efficient string concatenation by writing into an internal byte buffer that only emits the final string once. There's also strings.Reader, which wraps a string as an io.Reader for use with any Go API expecting a stream.- Search:
Contains,ContainsAny,HasPrefix,HasSuffix,Index,LastIndex,Count. - Split/Join:
Split,SplitN,SplitAfter,Fields(split on whitespace),Join. - Replace:
Replace,ReplaceAll,NewReplacerfor multi-pair replacement. - Trim:
TrimSpace,Trim,TrimLeft,TrimRight,TrimPrefix,TrimSuffix. - Case:
ToLower,ToUpper,Title(deprecated; usecases.Title),EqualFold(case-insensitive equality). - Build/transform:
Repeat,Map(rune mapper),NewReader. - Assembly:
strings.BuilderwithWriteString,WriteRune,WriteByte, thenString().
+ creates a brand new string and copies both operands. Concatenating n strings of average length k with += in a loop is O(n² × k) — building a 10,000-character string from pieces can take seconds. With strings.Builder, the buffer grows in geometric jumps (like a slice), making the total cost O(n × k) — typically 100× to 1000× faster on real workloads.- Free functions, not methods. Unlike Python (
s.split()) or Java (s.toLowerCase()), Go uses package-level functions:strings.Split(s, ","),strings.ToLower(s). This is consistent with Go's "no method call without a receiver type" rule. strings.Buildervs JavaStringBuilder: Same idea, slightly different API — Go usesWriteString/WriteRune/WriteByteinstead ofappend.- vs Python
"".join(list): The same trick.strings.Join(slice, ",")is the most direct equivalent. - vs
bytespackage: Thebytespackage mirrorsstringsfor[]byte— same function names, same semantics. Use it when working with binary data.
strings package first (instead of writing your own loop or regex) keeps code idiomatic, fast, and free of subtle Unicode bugs. strings.Builder is the go-to for building strings in loops, log line assembly, JSON-like serialization, code generation, and template rendering. Combined with fmt.Fprintf(builder, "...", ...), you get efficient typed formatting without intermediate allocations.- Don't copy a Builder. Once you've called
WriteString, copying invalidates the buffer. Use a pointer if you must pass it around. - Pre-grow with
Builder.Grow(n)when you know the output size — avoids repeated re-grows. - For tiny known concatenations (2–3 fixed strings), plain
+is fine and the compiler will optimize. The Builder pays off in loops or when assembling many parts.
The strings package provides all string manipulation functions. For efficient concatenation in loops, always use strings.Builder instead of +.
import "strings"
// Essential string functions
strings.Contains("Hello Go", "Go") // true
strings.Count("Hello", "l") // 2
strings.HasPrefix("Hello", "He") // true
strings.HasSuffix("Hello", "lo") // true
strings.ToUpper("hello") // "HELLO"
strings.ToLower("HELLO") // "hello"
strings.TrimSpace(" hi ") // "hi"
strings.Repeat("Go ", 3) // "Go Go Go "
strings.Replace("Hello Go", "Go", "World", 1) // "Hello World"
strings.ReplaceAll("aabaa", "a", "x") // "xxbxx"
// Split and Join
parts := strings.Split("apple,orange,banana", ",") // ["apple","orange","banana"]
joined := strings.Join(parts, " - ") // "apple - orange - banana"
// SplitN — split with limit
fmt.Println(strings.SplitN("a=b=c=d", "=", 2)) // ["a", "b=c=d"]
fmt.Println(strings.SplitN("a=b=c=d", "=", 3)) // ["a", "b", "c=d"]
// String conversion
num := 18
str := strconv.Itoa(num) // int to string: "18"
fmt.Println(len(str)) // 2 (character count of "18")
strings.Builder (efficient concatenation)
// strings.Builder avoids repeated allocations in loops
// Using + in a loop creates a NEW string each time = O(n²)
// Builder writes to a buffer = O(n)
var builder strings.Builder
builder.WriteString("Hello") // append string
builder.WriteString(", ") // append more
builder.WriteString("world!") // append more
result := builder.String() // "Hello, world!" — convert to string
builder.WriteRune(' ') // append a single rune/character
builder.WriteString("How are you")
builder.Reset() // clear the builder for reuse
builder.WriteString("Starting fresh!")
fmt.Println(builder.String()) // "Starting fresh!"
- HasPrefix for routing —
strings.HasPrefix(r.URL.Path, "/api/v2/")for version-based routing - Split for CSV / log parsing — quick parse of
k=vpairs, log lines, env files - strings.Builder for templates — building HTML/SQL programmatically without O(n²) allocation
- strings.NewReader — wraps a string as an
io.Readerfor tests, JSON decoding - EqualFold — case-insensitive comparison without allocating; used in HTTP header comparisons
- strings.Cut (Go 1.18+) — replaces SplitN(2) for parsing
"key=value":k, v, ok := strings.Cut(s, "=")
15 Structs
type User struct { ID int; Name string; Email string }. Structs are Go's equivalent of a class — but without methods inside the type body. Methods are declared separately with an explicit receiver. Structs can be:
- Named (the usual case): declared with
type X struct { ... }. - Anonymous: declared inline as part of a variable or function signature.
- Nested: a struct inside a struct.
- Embedded: a field with no name, promoting the inner type's fields and methods.
- Tagged: with backtick metadata strings used by encoders, validators, and ORMs.
- Zero value:
var u User— every field is the type's zero value. This is immediately usable in Go ("useful zero values" is a core philosophy). - Positional literal:
u := User{1, "Yatin", "y@example.com"}— fragile if fields are reordered, almost never used in production. - Named literal:
u := User{ID: 1, Name: "Yatin"}— preferred form; missing fields get zero values; resilient to field additions. - Pointer literal:
u := &User{ID: 1, Name: "Yatin"}— get a heap-allocated pointer in one expression. - Constructor function:
u := NewUser(1, "Yatin", "y@example.com")— convention isNewXreturning*XorX.
- vs OOP classes (Java/C#): No inheritance — only composition (embedding). No constructors — use a
NewXfunction if you need one. No methods inside the body — declared separately withfunc (r Receiver) Method(). Nothis/self— the receiver name is whatever you choose. - Value types by default: Assignment, function arguments, and map values copy the entire struct. Use a pointer (
*User) if you want sharing or mutation. - vs C structs: Same memory layout, but with methods, tags, and embedded fields. No bit fields (use
uint8+ bitwise ops manually). - vs Rust structs: Very similar — both are value types with separate
implblocks for methods. Go skips Rust's lifetime annotations and traits. - Comparable structs: Two structs are
==if all fields are==. Structs containing slices/maps/funcs are not comparable.
User, Order, Config, HTTPResponse) cleanly and efficiently. The "useful zero value" philosophy avoids null-pointer exceptions and constructor boilerplate — var b bytes.Buffer is immediately usable. Field tags drive Go's entire serialization story — JSON, XML, SQL, YAML, env-var binding, request validation — without sprinkling reflection or annotation magic in your code. The lack of inheritance forces composition, which over time produces flatter, more refactorable code.struct { a bool; b int64; c bool } takes 24 bytes on a 64-bit machine, while struct { b int64; a bool; c bool } takes only 16 bytes — a 33% savings just by ordering fields from largest to smallest. Run fieldalignment linter to find waste, especially important in hot loops or large in-memory caches.type User struct { // groups related data together
ID int // unique identifier
FirstName string
LastName string
Email string
Age int
Active bool // zero value is false (safe default)
}
// Create
u1 := User{ID: 1, FirstName: "Yatin", Email: "y@test.com", Active: true} // named fields
u2 := &User{FirstName: "Ptr"} // pointer to struct
u3 := new(User) // *User, all zero values
// Access — auto-dereferences pointers
u2.FirstName // same as (*u2).FirstName
// Struct tags (json, db, validate)
type Product struct {
ID int `json:"id" db:"product_id"` // maps to JSON key "id"
Name string `json:"name" validate:"required"` // validation tag
Price float64 `json:"price,omitempty"` // omit from JSON if zero
SKU string `json:"-"` // excluded from JSON
}
// Constructor pattern (Go has no constructors)
func NewUser(first, last, email string) *User { // returns pointer
return &User{FirstName: first, LastName: last, Email: email, Active: true} // set defaults here
}
// Anonymous struct
point := struct{ X, Y int }{10, 20} // one-off struct, no type name
_ = u1; _ = u3; _ = point // suppress unused-var errors
&Struct{} (pointer) vs Struct{} (value)
// VALUE — creates a copy on the stack
user := User{Name: "John", Age: 25} // type is User (value)
// POINTER — creates on heap, returns memory address
user2 := &User{Name: "Jane", Age: 30} // type is *User (pointer)
// ---- What happens with VALUE (Struct{}) ----
func updateAge(u User) {
u.Age = 99 // modifies the COPY, not the original!
}
user := User{Name: "John", Age: 25}
updateAge(user)
fmt.Println(user.Age) // still 25! The change was LOST
// ---- What happens with POINTER (&Struct{}) ----
func updateAgePtr(u *User) {
u.Age = 99 // modifies the ORIGINAL
}
user2 := &User{Name: "Jane", Age: 30}
updateAgePtr(user2)
fmt.Println(user2.Age) // 99! Change persisted
func NewUser(name string) *User {
return &User{Name: name, Active: true}
}
- Returns a pointer so callers can modify the struct through methods
- Avoids copying large structs (pointer = 8 bytes, struct could be 100s of bytes)
- Allows methods with pointer receivers to work
- Allows returning
nilto indicate failure or "no value"
Rule of thumb: Use &Struct{} (pointer) when you need to modify the struct or it's large. Use Struct{} (value) for small, immutable data (like coordinates, config constants).
- Domain models — every entity (User, Order, Product) is a struct, often with JSON+DB tags
- Request/response DTOs — separate
CreateUserRequeststruct from theUserdomain model — input validation lives on the DTO - Config structs — load YAML/env into a
Configstruct via libraries likeviperorenvconfig - Functional options pattern — pass
...OptiontoNewServerinstead of 10 positional args - Anonymous structs in tests — table-driven tests use
[]struct{name string; want int}{ ... } - Embedded structs —
type Admin struct { User; Permissions []string }— User fields are promoted
16 Struct Tags Deep Dive
Name string `json:"name" validate:"required" db:"user_name"`. Tags are invisible to normal Go code — they don't affect compilation or runtime semantics directly. Libraries read them via the reflect package at runtime to control marshaling (JSON/XML/YAML), validation, ORM column mapping, environment variable binding, CLI flag binding, OpenAPI generation, and more.- Format:
`key1:"value1" key2:"value2,option1,option2"` - Multiple keys separated by spaces, allowing one field to carry tags for several libraries simultaneously.
- Within a value, comma-separated options refine behavior (e.g.
omitempty,required,min=1). - Reading manually:
reflect.TypeOf(x).Field(i).Tag.Get("json")returns the value for that key.
json: standard library —`json:"name,omitempty"`,`json:"-"`to exclude.xml: standard library —`xml:"name,attr"`for attributes.yaml: viagopkg.in/yaml.v3.db: viajmoiron/sqlxfor SQL row mapping.gorm: via GORM ORM —`gorm:"primaryKey;autoIncrement"`.validate: via go-playground/validator —`validate:"required,email,max=100"`.form,uri,header: via Gin framework for binding HTTP request parts.env: via caarlos0/env for environment variable binding.flag: for CLI flag binding.protobuf: generated byprotoc-gen-go.
- vs Java/C# annotations: Java uses typed annotations like
@JsonProperty("name")with their own classes and runtime support. Go tags are just strings — much lighter, but less type-safe. - vs Python decorators: Python decorators wrap functions/classes with code. Go tags carry only metadata, never code.
- vs Rust attributes: Rust has
#[serde(rename = "name")]which is closer to Java annotations. Go's strings are simpler but require external parsing. - Multiple libraries can share one field:
`json:"id" db:"user_id" validate:"min=1"`— each consumer reads only its own key.
- Unexported fields are skipped by all encoders, regardless of tags.
name string `json:"name"`won't appear in JSON output. - Typos in tags are silent.
`josn:"name"`compiles fine but won't be picked up. Usego vet'sstructtagcheck. omitemptyrules vary by encoder. JSON considers zero values "empty"; not all encoders behave the same.- Tag strings must be a single line — multiline tags don't compile.
Struct tags are metadata strings attached to struct fields. They don't affect Go code directly — libraries use reflect to read them at runtime. Most commonly used for JSON, XML, database mapping, and validation.
type Person struct {
FirstName string `json:"first_name" db:"firstn" xml:"first"`
// json:"first_name" → JSON key becomes "first_name"
// db:"firstn" → database column name
// xml:"first" → XML element name
LastName string `json:"last_name,omitempty"`
// omitempty → exclude from output if zero value ("", 0, nil, false)
Age int `json:"-"`
// "-" → NEVER include in JSON output (hide sensitive data)
}
person := Person{FirstName: "Jane", Age: 30} // LastName is empty
jsonData, _ := json.Marshal(person)
fmt.Println(string(jsonData))
// {"first_name":"Jane"}
// Note: last_name omitted (empty + omitempty), Age omitted (json:"-")
| Tag | Meaning |
|---|---|
json:"name" | Rename field in JSON |
json:",omitempty" | Omit if zero value |
json:"-" | Always exclude from JSON |
xml:"name,attr" | XML attribute instead of element |
db:"column_name" | Database column mapping |
validate:"required" | Validation rules |
binding:"required,email" | Gin framework validation |
- API responses —
json:"created_at"converts Go'sCreatedAtto snake_case for JS clients - Hide secrets —
Password string `json:"-"`ensures it never leaks to API responses - SQL ORM mapping —
db:"user_id"tellssqlx/gormwhich DB column maps to which struct field - Validation —
validate:"required,email,min=8"enforced by go-playground/validator (used inside Gin) - Multiple targets — same struct can have json+db+validate+xml tags simultaneously, reused across the layers
- OpenAPI generation — tools like
swagread tags + comments to auto-generate API docs
17 Pointers
& takes the address (p := &x gives a *int), and * dereferences (*p = 100 writes through the pointer). *T is the type "pointer to T". The zero value of any pointer is nil. Pointers serve two purposes in Go: they let functions mutate their arguments, and they let you avoid copying large values when passing them around.- Address-of:
p := &x—phas type*intifxisint. - Dereference (read):
v := *p— copies the valueppoints at. - Dereference (write):
*p = 100— writes through the pointer. - New:
p := new(int)— allocates a zero-valuedinton the heap and returns*int. - Struct literal pointer:
u := &User{Name: "Yatin"}— equivalent to taking the address of a struct literal. - Method auto-deref:
p.Method()automatically dereferencespif needed; you almost never write(*p).Method(). - Field auto-deref:
p.Nameworks whetherpisUseror*User. - Nil check:
if p == nil { ... }— dereferencing a nil pointer panics.
- vs C/C++: Go has no pointer arithmetic —
p++is illegal. This eliminates buffer overruns, the most common C memory bug. The GC handles cleanup, so no manualfree.unsafeexists as an opt-in escape hatch. - vs Java/Python/JS: In those languages every object is implicitly a reference; you can't choose. Go gives you the choice:
User(value, copied on assignment) vs*User(pointer, shared). This explicitness is a big win for performance reasoning. - vs Rust references: Rust enforces lifetimes and aliasing rules at compile time; Go relies on the GC to keep things safe. Easier to learn, slightly slower.
- Stack vs heap: Go's escape analysis decides automatically — if a pointer "escapes" the function (returned, stored in a global, captured by a goroutine), the value lives on the heap; otherwise on the stack.
func (u User) X()) or pointer receiver (func (u *User) X()). This is one of Go's most important design decisions:
- Use a pointer receiver if: the method needs to mutate the receiver, the struct is large (avoid copies), the type contains a
sync.Mutex, or you want consistency across all methods of the type. - Use a value receiver if: the type is small (a few words), it's immutable by nature (e.g.
time.Time), and you don't need mutation. - Don't mix: if any method on a type uses a pointer receiver, all methods should — otherwise the method set rules get confusing for interface satisfaction.
- Nil dereference panic:
var p *User; p.Namecrashes. Always check or guarantee non-nil. - Address of map element:
&m["key"]is illegal — map values are not addressable. - Loop variable capture (pre-Go 1.22):
for _, x := range slice { go func() { use(&x) }() }— all goroutines see the same address. Fixed in Go 1.22. - Nil interface vs nil pointer: An interface holding a typed nil pointer is itself not nil.
var p *MyError = nil; var e error = p; e == nil→false! Famous footgun.
x := 42 // regular variable on stack
p := &x // p is *int, points to x
fmt.Println(*p) // 42 (dereference)
*p = 100 // write through the pointer
fmt.Println(x) // 100 — changed through pointer!
// nil pointer
var np *int // nil
// *np → PANIC: nil pointer dereference!
// Go has NO pointer arithmetic (unlike C)
// Pass by value vs pointer
func doubleValue(n int) { n *= 2 } // copy, original unchanged
func doublePtr(n *int) { *n *= 2 } // modifies original
// When to use pointers:
// 1. Mutate the argument
// 2. Large structs (avoid copying)
// 3. Represent "absence" (nil)
// When NOT: small structs, slices/maps/channels (already refs)
_ = np // suppress unused-var error
- Mutating method receivers —
func (u *User) SetEmail(e string)modifies the actual user, not a copy - Optional fields in JSON —
*string/*intdistinguishes"field":null, missing field, and"field":"" - Avoiding large struct copies — passing a 200-byte struct by value to a hot function wastes CPU; pointer is 8 bytes
- Linked data structures — trees, linked lists, graphs — pointers are essential
- Slices/maps/channels — already reference types, no need to take the address
- nil checks before dereferencing —
if u != nil { u.Save() }guards against nil pointer panics
18 Methods & Interfaces
A method is a function with an explicit receiver placed between
func and the function name: func (r Rectangle) Area() float64 { ... }. The receiver makes the function "belong to" the type — you call it as rect.Area(). Methods can be defined on any named type in your package, not just structs: type Celsius float64; func (c Celsius) Fahrenheit() float64 { ... }.An interface is a set of method signatures:
type Stringer interface { String() string }. Any type that has all the listed methods automatically satisfies the interface — there's no implements keyword. This is called structural typing, and it's one of Go's most distinctive features.- Value receiver:
func (r Rectangle) Area()— gets a copy of the receiver. Cannot mutate the original. Cheap for small types. - Pointer receiver:
func (r *Rectangle) Scale(f float64)— gets a pointer, can mutate the original, avoids copying for large structs. - Auto-conversion: calling
p.Method()whereMethodhas a pointer receiver works whetherpisTor*T(Go inserts&automatically) — but only if the value is addressable. - Method set rules for interface satisfaction: if interface methods include any pointer-receiver methods, you need a
*T, not aT, to satisfy the interface.
- Definition:
type Reader interface { Read(p []byte) (n int, err error) } - Empty interface:
interface{}(or its aliasanyin Go 1.18+) — holds any value. Pre-generics, this was the universal "any type" placeholder. - Type assertion:
v, ok := i.(Concrete)— extract the underlying type. - Type switch:
switch v := i.(type) { case int: ... case string: ... }— branch on the dynamic type. - Embedding interfaces:
type ReadWriter interface { Reader; Writer }— composes the method sets. - Internal representation: an interface value is a 2-word pair:
{type info, data pointer}. The famous "nil interface vs interface holding nil" footgun comes from the type-info word being non-nil.
- vs Java/C# nominal typing: Interface satisfaction is structural / implicit — you never write
implements Stringer. If your type has the right method, it satisfies. Java requires explicit declaration of implemented interfaces, which creates tight coupling. - vs duck typing in Python/Ruby: Same idea ("if it walks like a duck...") but Go checks at compile time, not runtime. You get the flexibility without the runtime errors.
- vs Rust traits: Rust traits are nominal (you must
impl Trait for Type). Go interfaces are structural — more flexible but less explicit. - vs C++ virtual methods: No vtable on the type itself; the interface value carries the method table at runtime, allowing the same type to satisfy unrelated interfaces.
- Methods on non-struct types: Unique among mainstream languages —
type MyInt int; func (m MyInt) Double() MyIntis perfectly valid.
type UserStore interface { Get(id int) (*User, error) }), and the implementation doesn't have to import or know about you. This is the cleanest expression of "depend on abstractions, not implementations" in any mainstream language.Interfaces in Go should be small — the standard library famously favors single-method interfaces (
io.Reader, io.Writer, fmt.Stringer, http.Handler) that compose into bigger ones via embedding.io.Reader— anything you can read bytes from (file, socket, HTTP body, in-memory buffer).io.Writer— anything you can write bytes to.error— any type withError() string.fmt.Stringer— controls how a value prints with%v.http.Handler— anything that can serve an HTTP request.sort.Interface— pre-generic sortable collection (Len,Less,Swap).
Methods
type Rectangle struct { Width, Height float64 } // simple data struct
// Value receiver — gets a COPY
func (r Rectangle) Area() float64 { return r.Width * r.Height } // read-only, safe
// Pointer receiver — can MODIFY the struct
func (r *Rectangle) Scale(factor float64) { // r points to original
r.Width *= factor // mutate in-place
r.Height *= factor
}
// Rule: if ANY method needs pointer receiver, use pointer for ALL
Interfaces
// Interface = set of method signatures
// IMPLICITLY satisfied (no "implements" keyword)
type Shape interface { // any type with these methods satisfies Shape
Area() float64 // must implement Area
Perimeter() float64 // must implement Perimeter
}
type Circle struct { Radius float64 } // concrete type
func (c Circle) Area() float64 { return math.Pi * c.Radius * c.Radius } // π*r²
func (c Circle) Perimeter() float64 { return 2 * math.Pi * c.Radius } // 2πr
// Polymorphism
func PrintInfo(s Shape) { // accepts any Shape implementation
fmt.Printf("Area: %.2f\n", s.Area()) // calls correct method at runtime
}
// Empty interface — accepts ANY type
func printAnything(v any) { fmt.Println(v) } // any = interface{}
// Type assertion
s, ok := i.(string) // safe assertion
// Key stdlib interfaces:
// fmt.Stringer → String() string
// error → Error() string
// io.Reader → Read(p []byte) (n int, err error)
// io.Writer → Write(p []byte) (n int, err error)
// http.Handler → ServeHTTP(ResponseWriter, *Request)
// Interface composition
type ReadWriter interface { io.Reader; io.Writer } // embed two interfaces
// Best practice: accept interfaces, return structs
_ = s; _ = ok // suppress unused-var errors
An interface value is nil only if BOTH the type and value are nil:
var s *MyStruct = nil // typed nil pointer
var i Shape = s // interface holds (type=*MyStruct, value=nil)
fmt.Println(i == nil) // false! type is *MyStruct, value is nil
- Repository interfaces —
UserRepo interface { FindByID(id int) (*User, error) }— swap implementations (Postgres, in-memory test fake) without changing handler code - io.Reader / io.Writer — most powerful interfaces in Go: files, sockets, buffers, gzip — all interchangeable
- http.Handler — single-method interface: any type with
ServeHTTPworks as a route - Stringer for logging — implement
String() stringon your enum/status type for clean log output - "Accept interfaces, return structs" — function params take interfaces (flexibility); returns are concrete (predictability)
- Mocking for tests — define an interface in the consumer, mock it in tests with a fake struct
19 Embedding & Composition
type Dog struct { Animal; Breed string }. The outer type then promotes the embedded type's fields and methods, so you can call dog.Speak() as if Speak were defined on Dog directly. Internally, the embedded value is just a regular field whose name is the type name (dog.Animal.Speak() still works), but the promoted form is what makes embedding feel like inheritance.- Struct in struct:
type Dog struct { Animal; Breed string }— promotes fields and methods. - Interface in interface:
type ReadWriter interface { Reader; Writer }— composes method sets. - Interface in struct:
type LoggingReader struct { io.Reader; logger *Logger }— wraps a value satisfying the interface, useful for decorators/middleware.
dog.Speak()callsDog's version (if defined) or falls back toAnimal's.dog.Animal.Speak()always callsAnimal's version explicitly.- This is the closest Go gets to a
supercall. - If two embedded types have the same method name, you must call them via the explicit form to disambiguate — there's no automatic resolution.
- vs class inheritance (Java/C++/Python): Embedding looks like inheritance but isn't. It's "has-a" with method promotion. There's no virtual dispatch on the outer type, no
superkeyword, no protected/private hierarchy, no "is-a" subtype relationship. - The embedded type knows nothing about the outer. A method defined on
Animalcannot call methods overridden onDog— there's no polymorphism the way Java provides. - vs traits/mixins: Closer to Rust traits or Scala mixins than to classical inheritance. Pure composition, no fragile base class problem.
- vs Object.assign / spread: JS-style copying mixes properties at runtime; Go's embedding is a compile-time structural feature.
x.Method() without thinking about which "parent" defined it) with the safety of composition (no fragile base class, no diamond problem, no virtual-dispatch surprises). Embedding interfaces is how you build "interface unions" like io.ReadWriter = io.Reader + io.Writer — and Go's entire I/O ecosystem rests on this pattern. Embedding a struct that implements an interface lets your wrapper type inherit the interface satisfaction for free, which is the core of Go's decorator/middleware pattern.http.ServeMux: embeds async.Mutexdirectly so methods can callm.Lock()as if it were their own.- Middleware decoration:
type LoggingHandler struct { http.Handler }wraps any handler, overridesServeHTTP, and can call the embedded version viah.Handler.ServeHTTP(w, r). - Test fakes: embed an interface in a test struct, override only the methods you care about, the rest will panic on call (
nil pointer) — explicit signal that the test exercised the wrong path. - Domain hierarchies:
type Admin struct { User; Permissions []string }— gets allUsermethods plus its own.
// Go has NO inheritance. Uses COMPOSITION via embedding.
type Animal struct { Name, Sound string } // base struct
func (a Animal) Speak() string { return a.Name + " says " + a.Sound } // base method
type Dog struct {
Animal // embedded — promotes all fields & methods
Breed string // Dog-specific field
}
d := Dog{Animal: Animal{"Rex", "Woof"}, Breed: "German Shepherd"} // init embedded struct
d.Name // "Rex" (promoted from Animal)
d.Speak() // "Rex says Woof"
// Override methods
func (d Dog) Speak() string { return d.Name + " barks!" } // shadows Animal.Speak
- http.Server embedding — your custom server can embed
*http.Serverand add middleware without inheritance - Mutex embedding —
type SafeMap struct { sync.Mutex; m map[string]int }exposesLock/Unlockdirectly - Repository pattern — embed a base repo with common CRUD, add domain-specific methods on the wrapper
- Interface embedding —
io.ReadWriterembedsReaderandWriter; you compose interfaces, not classes - Decorator pattern — embed an interface and override one method to wrap behavior (e.g., add logging to a Logger)
- Avoid deep nesting — Go doesn't have inheritance for a reason; 2 levels deep is usually the max
20 Error Handling
error is a tiny interface — any type with an Error() string method satisfies it. Functions that can fail return an extra error value as their last return, and callers check it explicitly with if err != nil. There is no try, no catch, no throw, no stack unwinding — errors flow through ordinary return values.error interface and creation
- The interface:
type error interface { Error() string }— that's the entire definition. - Static text errors:
errors.New("not found")— produces an immutable error value. - Sentinel errors: package-level
var ErrNotFound = errors.New("not found")— comparable with==orerrors.Is. - Formatted errors:
fmt.Errorf("user %d: %w", id, err)— the%wverb wraps another error to preserve the chain. - Custom error types: any struct implementing
Error() string; can carry rich context (HTTP status, retry hints, fields).
errors.Is(err, target)— walks the wrap chain, returns true if any link equalstarget. Use this for sentinel comparisons:errors.Is(err, sql.ErrNoRows).errors.As(err, &target)— walks the chain looking for an error of typetarget; if found, sets it. Used for typed errors carrying data.errors.Unwrap(err)— returns the next error in the chain (or nil).errors.Join(errs...)(Go 1.20+) — combines multiple errors into one.
- vs Python/Java/JS exceptions: No try/catch. Errors don't unwind the stack — they flow through return values like any other data. Every line is not a potential throw site, which makes control flow predictable.
- vs Rust
Result<T, E>: Same idea — error as value — but Rust uses an enum and the?operator to bubble up. Go has no equivalent of?; you writeif err != nil { return ..., err }by hand. - vs Go
panic:panicexists for unrecoverable bugs (nil deref, index out of range, programmer assertion failures). It is not the normal error mechanism — using it for ordinary failures is considered very un-Go. - The famous verbosity: Yes, you'll write
if err != nil { return err }a lot. The trade-off is that every error site is visible in the source — no surprise exceptions, no missed handlers.
%w preserves the chain so you can match a deep error (errors.Is(err, sql.ErrNoRows)) without losing the high-level context. The result is code where error paths are fully traceable from the call site to the source.- Wrap with context at each layer:
fmt.Errorf("loading user %d: %w", id, err). Don'treturn errbare from deep call chains. - Use sentinel errors for "expected" failure modes the caller should handle (
io.EOF,sql.ErrNoRows). - Use typed errors when you need to carry data (HTTP status code, validation field name).
- Don't log AND return the same error — pick one. Logging at the top of the call stack avoids duplicate noise.
- Compare with
errors.Is, not==, in modern code so wrapping doesn't break checks. - Reserve
panicfor "programmer error" situations: package init failures, impossible-state assertions.
// error is an interface: Error() string
err1 := errors.New("something went wrong") // simple static error
err2 := fmt.Errorf("user %d not found", 42) // formatted error message
// The Go pattern: always check errors
data, err := os.ReadFile(path) // returns ([]byte, error)
if err != nil { // nil means success
return nil, fmt.Errorf("reading %s: %w", path, err) // %w wraps
}
// Custom error types
type ValidationError struct { Field, Message string } // rich error info
func (e *ValidationError) Error() string { // satisfies error interface
return fmt.Sprintf("%s: %s", e.Field, e.Message)
}
// Sentinel errors
var ErrNotFound = errors.New("not found") // compare with errors.Is
// errors.Is — check through wrapping chain
if errors.Is(err, ErrNotFound) { /* handle */ } // works through wrapped errors
// errors.As — extract specific error type
var ve *ValidationError // target type to extract
if errors.As(err, &ve) { fmt.Println(ve.Field) } // ve populated if matched
_ = err1; _ = err2; _ = data // suppress unused-var errors
- Error wrapping with
%w—fmt.Errorf("loading user %d: %w", id, err)preserves the chain so handlers upstream canerrors.Isthe root cause - Sentinel errors —
sql.ErrNoRows,io.EOF,context.Canceled— compared witherrors.Is - Custom error types —
ValidationError,NotFoundError—errors.Asextracts them in HTTP middleware to map to status codes - Error groups —
golang.org/x/sync/errgroupfor concurrent operations: first error cancels the rest - Don't
log.Fatalin libraries — return errors so the caller decides; only main packages should exit - Stack traces —
github.com/pkg/errorsor Go 1.20+errors.Joinfor combining multiple errors
21 Type Conversion & Number Parsing
Type conversion changes a value's type when the underlying representation is compatible —
int(3.14), float64(x), string(byteSlice). The syntax is always TargetType(value). Conversions never fail (in the error-return sense) but may lose information (truncation, overflow).Number parsing turns a
string into a number using the strconv package — strconv.Atoi("42"), strconv.ParseFloat("3.14", 64). Parsing can fail and returns an error.Both are explicit — Go never silently coerces between types.
- String → int:
strconv.Atoi("42")→(42, nil). The shortcut forParseInt(s, 10, 0). - String → int with base/size:
strconv.ParseInt("ff", 16, 64)→(255, nil). - String → uint:
strconv.ParseUint("42", 10, 64) - String → float:
strconv.ParseFloat("3.14", 64) - String → bool:
strconv.ParseBool("true")— accepts "1", "t", "T", "TRUE", "true", "True", and the falses. - Int → string:
strconv.Itoa(42)→"42". Shortcut forFormatInt(int64(i), 10). - Float → string:
strconv.FormatFloat(3.14, 'f', 2, 64)→"3.14". - Bool → string:
strconv.FormatBool(true)→"true". - Quote:
strconv.Quote("hi\n")→"\"hi\\n\""— Go-syntax string literal.
- vs JavaScript: JS has weak typing —
"5" + 1 === "51"and"5" * 1 === 5. Go won't compile either expression — you must explicitly callstrconv.Atoi("5")orfmt.Sprintf("%s%d", s, i). - vs Python: Python raises
TypeErrorat runtime for similar mistakes. Go catches them at compile time. - vs Java: Java has implicit widening (
int → long) but explicit narrowing. Go is fully explicit in both directions:int + int64doesn't compile. - Conversion syntax: Always
TargetType(value)— looks like a function call. There's nov as T(Rust/TS) or C-style(T)v. - Parsing returns an error, conversions don't — a critical distinction.
int(3.7)just truncates to3;strconv.Atoi("hi")returns an error.
int64 → int32) at a glance. Forcing parsing to return an error means user input always has a defined error path.string(65)is "A", not "65" — converting an int to string treats it as a Unicode code point.go vetwarns. Usestrconv.Itoa(65)for "65".- Float to int truncates, doesn't round:
int(3.9)→3. Usemath.Roundfirst if you need rounding. - Narrowing conversions silently overflow:
int8(300)→44, no panic, no warning. The race detector andgovet'sshiftcheck help. - String ↔ []byte converts copy memory by default. Use
unsafe.String/StringData(Go 1.20+) in hot paths to avoid the copy.
Type Conversion
Go has no implicit type conversion. You must explicitly convert using Type(value) syntax. This prevents subtle bugs that plague languages with automatic coercion.
// Syntax: TargetType(value)
var a int = 32
b := int32(a) // int → int32
c := float64(b) // int32 → float64 (32.0)
e := 3.14
f := int(e) // float64 → int (3, truncates decimal!)
// String ↔ Byte slice
g := "Hello @ こんにちは 🧑 привет"
h := []byte(g) // string → byte slice (UTF-8 bytes)
fmt.Println(h) // [72 101 108 108 111 32 64 32 ...]
i := []byte{255, 120, 72}
j := string(i) // byte slice → string
// CANNOT convert between unrelated types
// d := bool("true") // COMPILE ERROR! Use strconv.ParseBool
Number Parsing (strconv package)
Use the strconv package to convert between strings and numbers. Always handle the error — invalid input is common.
import "strconv"
// String → Int
num, err := strconv.Atoi("12345") // Atoi = "ASCII to Integer"
fmt.Println(num + 1) // 12346
// ParseInt — control base and bit size
val, _ := strconv.ParseInt("12345", 10, 32) // base 10, fit in int32
// String → Float
pi, _ := strconv.ParseFloat("3.14", 64) // 64 = float64 precision
fmt.Printf("Parsed float: %.2f\n", pi) // 3.14
// Binary string → Decimal
decimal, _ := strconv.ParseInt("1010", 2, 64) // base 2
fmt.Println(decimal) // 10 (0+2+0+8)
// Hex string → Decimal
hex, _ := strconv.ParseInt("FF", 16, 64) // base 16
fmt.Println(hex) // 255
// Int → String
s := strconv.Itoa(42) // "42"
// Error handling — invalid input
_, err = strconv.Atoi("456abc") // fails!
if err != nil {
fmt.Println("Error:", err) // strconv.Atoi: parsing "456abc": invalid syntax
}
- URL query params —
page, _ := strconv.Atoi(r.URL.Query().Get("page"))for paginated APIs - Env var parsing —
strconv.ParseBool(os.Getenv("DEBUG")), ParseInt for ports/timeouts - CSV/log line parsing — converting string fields to typed values for analytics pipelines
- []byte ↔ string — JSON marshaling returns
[]byte; convert withstring(data)for printing - Float truncation gotcha —
int(3.99) = 3, not 4 — usemath.Round()if you want rounding - Use ParseFloat for money input — never trust user-typed numbers; always handle parse errors
22 Fmt Package & Formatting Verbs
fmt package: formatted I/O modeled after C's printf/scanf but type-safe and reflection-aware. The package centers on formatting verbs — codes like %d, %s, %v, %+v — that tell the formatter how to render each argument. The function family is consistent: prefix-less names (Print, Printf) write to stdout; S-prefixed (Sprint, Sprintf) return a string; F-prefixed (Fprint, Fprintf) write to any io.Writer; Errorf returns an error.Print/Println/Printf→ write toos.StdoutSprint/Sprintln/Sprintf→ return astringFprint/Fprintln/Fprintf→ write to anyio.Writer(file, network, buffer)Errorf→ returns anerror(the modern way to construct wrapped errors with%w)Scan/Scanln/Scanf→ read from stdin (rarely used;bufio.Scanneris more common)Sscan/Sscanf→ parse from a string
- General:
%vdefault format,%+vwith field names,%#vGo-syntax,%Ttype name,%%literal percent. - Boolean:
%t→ "true"/"false". - Integer:
%ddecimal,%bbinary,%ooctal,%x/%Xhex,%cUnicode char,%UUnicode point. - Float:
%f/%Fdecimal,%e/%Escientific,%g/%Gshortest,%.2fprecision. - String:
%sraw,%qdouble-quoted,%xhex bytes. - Pointer:
%phex pointer address. - Errors:
%w— wraps an error so it can be unwrapped witherrors.Unwrap/Is/As. Only valid infmt.Errorf. - Width and padding:
%10dright-aligned in 10 cols,%-10dleft-aligned,%010dzero-padded.
- vs C printf: Go verbs are type-checked by
go vet— passing the wrong arg type for a verb is caught before the program runs. C's printf is happy to crash at runtime if you mismatch%dwith a pointer. - vs Python f-strings: Python's f-strings are nicer for inline interpolation but only work at compile time.
fmt.Sprintfworks on dynamic format strings. - The
%vverb is unique: it inspects the value's type via reflection and renders it sensibly — works on structs, slices, maps, channels, even functions. - Custom formatting: implement
fmt.Stringer(String() string) to control how your type prints with%vand%s. Implementfmt.Formatterfor full control. - Error wrapping (
%w) is unique — it builds the error chain thaterrors.Is/errors.Aswalk.
fmt.Printf("%+v\n", x)), error construction (fmt.Errorf), and string building (fmt.Sprintf) — all with the same consistent API. Mastering the verbs makes log lines, error messages, and quick debug prints feel effortless. The go vet integration means typos get caught instantly.fmt uses reflection and is therefore slower than direct concatenation for hot paths. For benchmark-driven code, prefer strconv + strings.Builder (or bytes.Buffer) for assembly. For structured logging, prefer slog over fmt.Sprintf + log call. fmt.Println alone allocates an empty interface slice — measurable in tight loops.Printing Functions
// Print — no newline, no spaces between args
fmt.Print("Hello ")
fmt.Print("World!") // HelloWorld! (same line)
// Println — adds newline, spaces between args
fmt.Println("Hello", "World!") // Hello World!\n
// Printf — formatted output (like C's printf)
name := "John"
age := 25
fmt.Printf("Name: %s, Age: %d\n", name, age)
fmt.Printf("Binary: %b, Hex: %X\n", age, age) // 11001, 19
// Sprint/Sprintf — return string instead of printing
s := fmt.Sprint("Hello", "World!") // "HelloWorld!"
sf := fmt.Sprintf("Name: %s", name) // "Name: John"
// Errorf — create formatted errors
err := fmt.Errorf("Age %d is too young", 15)
// Scan — read user input
var input string
fmt.Scan(&input) // reads one word from stdin
fmt.Scanln(&input) // reads until newline
fmt.Scanf("%s", &input) // reads with format
Formatting Verbs Reference
| Category | Verb | Description | Example |
|---|---|---|---|
| General | %v | Default format | 42, [1 2 3] |
%#v | Go-syntax format | main.User{Name:"John"} | |
%T | Type of value | int, string | |
%% | Literal percent | % | |
| Integer | %d | Decimal (base 10) | 255 |
%b | Binary (base 2) | 11111111 | |
%o / %O | Octal / with 0o prefix | 377 / 0o377 | |
%x / %X | Hex lowercase/uppercase | ff / FF | |
%04d | Zero-padded width 4 | 0042 | |
| Float | %f | Decimal point | 3.140000 |
%.2f | 2 decimal places | 3.14 | |
%e | Scientific notation | 3.14e+00 | |
| String | %s | Plain string | Hello |
%q | Double-quoted | "Hello" | |
%10s | Right-justified width 10 | Hello | |
| Bool | %t | true/false | true |
| Pointer | %p | Memory address | 0xc0000b4008 |
// Padding and alignment
fmt.Printf("%05d\n", 424) // 00424 (zero-padded)
fmt.Printf("|%10s|\n", "Hello") // | Hello| (right-aligned)
fmt.Printf("|%-10s|\n", "Hello") // |Hello | (left-aligned)
// Raw strings vs interpreted strings
msg1 := "Hello \nWorld!" // \n = actual newline
msg2 := `Hello \nWorld!` // \n = literal text (raw string)
%vfor debugging — universal verb that works on any type;%+vshows struct field names;%#vshows full Go syntax%wfor error wrapping —fmt.Errorf("loading: %w", err)— only valid inErrorffmt.Sprintffor log lines — building structured log messages or filenames with timestamps%-20sfor tables — left-pad columns when printing CLI tables (kubectl-style output)%qfor safe quoting — escapes special characters when displaying user input in logsfmt.Stringer— implementingString() stringon a type makes%vauto-call it (used for enums, IDs)
23 Regular Expressions
regexp package — pattern matching and extraction over strings and byte slices. The workflow is: compile a pattern into a *regexp.Regexp (once), then call methods on it (MatchString, FindString, FindAllStringSubmatch, ReplaceAllString, etc.). Use regexp.MustCompile for known-good patterns at package init (panics on bad pattern), or regexp.Compile when the pattern comes from user input (returns an error). Patterns are written as raw string literals with backticks so you don't have to double-escape backslashes.(All)?(String)?(Submatch)?(Index)?:
- Match:
MatchString(s)→ just true/false. - Find:
FindString(s)→ first match.FindAllString(s, -1)→ all matches. - FindSubmatch:
FindStringSubmatch(s)→ match + capture groups as[]string. - FindIndex: returns byte offsets instead of strings — useful for replacement.
- Replace:
ReplaceAllString(s, "$1-$2"),ReplaceAllStringFunc(s, func(match string) string {...}). - Split:
Split(s, -1)— split on every match. - Named groups:
(?P<name>...)in pattern,SubexpIndex("name")to look up.
- Lost features: No backreferences (
\1in the pattern), no lookahead/lookbehind ((?=...),(?<=...)), no recursive subpatterns. - What you get in exchange: guaranteed linear time matching. RE2 cannot suffer "catastrophic backtracking" — the kind of pathological pattern that makes a PCRE-based service hang for minutes on a malicious input.
- Same syntax for common features: character classes
[a-z], anchors^$, quantifiers*+?{n,m}, alternation|, capture groups(...), non-capturing groups(?:...), named groups(?P<name>...), flags(?i). - Same trick as Rust's
regexcrate, which also descends from RE2.
*Regexp is safe for concurrent use by multiple goroutines, so the standard pattern is "compile once at package init, reuse everywhere". For pure prefix/suffix/contains checks, the strings package is much faster — only reach for regex when you really need pattern matching.- Always compile once. Compiling inside a loop is 100× slower than reusing a compiled pattern.
- Use raw strings (
backtick):`\d+`not"\\d+". - Anchor your patterns with
^and$when matching whole strings — much faster than scanning. - Don't regex when
stringsworks:strings.HasPrefixbeats^foo;strings.Containsbeatsfoo. - For performance-critical parsing (URLs, logs, structured text), hand-written parsers are usually 5–20× faster than regex.
Go's regexp package uses RE2 syntax (guaranteed linear time). Use MustCompile for known-good patterns, Compile when pattern might be invalid.
import "regexp"
// Compile a regex — MustCompile panics if pattern is invalid
re := regexp.MustCompile(`[a-zA-Z0-9._+%-]+@[a-zA-Z0-9.-]+\.[a-zA-Z]{2,}`)
// Match test
fmt.Println(re.MatchString("user@email.com")) // true
fmt.Println(re.MatchString("invalid_email")) // false
// Capturing groups with FindStringSubmatch
re = regexp.MustCompile(`(\d{4})-(\d{2})-(\d{2})`)
matches := re.FindStringSubmatch("2024-07-30")
fmt.Println(matches[0]) // "2024-07-30" (full match)
fmt.Println(matches[1]) // "2024" (group 1)
fmt.Println(matches[2]) // "07" (group 2)
fmt.Println(matches[3]) // "30" (group 3)
// Replace all matches
re = regexp.MustCompile(`[aeiou]`)
result := re.ReplaceAllString("Hello World", "*")
fmt.Println(result) // "H*ll* W*rld"
// Case-insensitive flag (?i)
re = regexp.MustCompile(`(?i)go`)
fmt.Println(re.MatchString("Golang is great")) // true
// FindAllString — find all occurrences
re = regexp.MustCompile(`\d+`) // match digits
all := re.FindAllString("Hello 123 Go 456!", -1) // -1 = find all
fmt.Println(all) // ["123", "456"]
- Input validation — email, phone, SKU, slug formats — but consider
net/mail.ParseAddressfor emails (more correct) - Log parsing — extract structured fields from nginx/syslog lines:
(\d+\.\d+\.\d+\.\d+) - .* "(GET|POST) (.*?) HTTP - Compile once — store
regexp.MustCompileresult as a package var; recompiling on every call is 100x slower - RE2 limitations — Go's regex has no backreferences (no
\1); guaranteed linear time, no catastrophic backtracking - Named captures —
(?P<year>\d{4})+SubexpNames()for self-documenting parsers - Templating with replace —
ReplaceAllStringFuncfor transforming each match (e.g., highlighting search terms in HTML)
24 Recursion
- Base case: the condition where the function returns directly without recursing — this is what stops the chain.
- Recursive case: the self-call with a strictly smaller input (or one closer to the base case).
- Direct recursion: a function calls itself directly.
factorial(n)→n * factorial(n-1). - Mutual recursion: two or more functions call each other.
isEven(n)callsisOdd(n-1)and vice versa. - Tail recursion: the recursive call is the very last action; the result is returned directly. Some languages optimize this into a loop — Go does not.
- Tree recursion: the function calls itself multiple times per call.
fib(n) = fib(n-1) + fib(n-2). Without memoization, exponential blowup. - Recursion with accumulator: pass partial results down to make tail-style recursion efficient.
- Backtracking: recursion + state mutation + undo. Sudoku solvers, N-queens, maze exploration.
- No tail-call optimization (TCO). Unlike Scheme, Scala, or Haskell, Go does not rewrite tail-recursive calls into loops. Deep tail-recursive Go code grows the stack and can panic with
stack overflow. - BUT goroutine stacks are dynamic. They start at ~2 KB and grow on demand up to
GOMAXSTACK(default 1 GB). This means Go can typically handle deeper recursion than fixed-stack threads in C/Java. - Mutual recursion just works: Go resolves names lazily within a package, so two functions can call each other without forward declarations.
- vs Python: Python's default recursion limit is 1000 — you'll
RecursionErrorlong before Go would. - vs JavaScript: JS has no TCO either (despite the spec) and a much smaller default stack — recursion depth often maxes out around 10,000.
- The recursion depth could be unbounded (input from user/network).
- You're seeing exponential time blowup without memoization.
- The hot path runs in a tight loop where call overhead matters.
- You're recursing only at the tail position — replace with a loop.
- Use an explicit stack (
[]Item) when you need recursion semantics but no stack risk.
A function that calls itself. Every recursive function needs a base case (when to stop) and a recursive case (call itself with a smaller problem).
// Classic: Factorial — n! = n × (n-1) × ... × 1
func factorial(n int) int {
if n == 0 { return 1 } // base case: 0! = 1
return n * factorial(n-1) // recursive case
}
// factorial(5) → 5 * 4 * 3 * 2 * 1 * 1 = 120
// Sum of digits: 12345 → 1+2+3+4+5 = 15
func sumOfDigits(n int) int {
if n < 10 { return n } // single digit = base case
return n%10 + sumOfDigits(n/10) // last digit + sum of rest
}
// sumOfDigits(12345):
// 5 + sumOfDigits(1234)
// 5 + 4 + sumOfDigits(123)
// 5 + 4 + 3 + sumOfDigits(12)
// 5 + 4 + 3 + 2 + sumOfDigits(1)
// 5 + 4 + 3 + 2 + 1 = 15
Unlike some languages, Go does NOT optimize tail-recursive functions. Deep recursion (~10,000+ calls) can cause stack overflow. For large inputs, prefer iterative (loop) solutions.
- Filesystem walk —
filepath.Walkuses recursion internally to traverse directory trees - JSON / YAML / XML traversal — recursing into nested
map[string]anyfor config validation or merging - AST processing —
go/astwalks the abstract syntax tree recursively (linters, code generators) - Tree algorithms — DOM parsers, BTree splits, expression evaluators
- Avoid for deep data — convert to iterative + explicit stack for files like
node_modules/with 100k+ entries - Memoization — wrap recursive calls in a
map[input]outputcache (DP-style)
25 Random Numbers
math/rand(and Go 1.22'smath/rand/v2) — fast, deterministic, pseudo-random numbers. Good for games, simulations, sampling, randomized testing, randomized backoff, picking a random element from a slice. NOT secure.crypto/rand— slower, cryptographically secure random bytes from the OS entropy source (/dev/urandomon Unix,CryptGenRandomon Windows). Use for passwords, session tokens, API keys, salts, nonces, anything an attacker shouldn't be able to predict.
crypto/rand goes through io.Reader so it composes with the rest of the standard library.
- Random int in [0, n):
rand.Intn(n) - Random int64:
rand.Int63() - Random float in [0, 1):
rand.Float64() - Shuffle a slice:
rand.Shuffle(len(s), func(i, j int) { s[i], s[j] = s[j], s[i] }) - Seed (only if you need reproducibility):
r := rand.New(rand.NewSource(42))— same seed always produces the same sequence. - Auto-seeding (Go 1.20+): the global functions are auto-seeded per program — no more
rand.Seed(time.Now().UnixNano())ritual.
- Random bytes:
b := make([]byte, 32); _, err := rand.Read(b) - Random integer in range:
n, err := rand.Int(rand.Reader, big.NewInt(100)) - Random hex token:
b := make([]byte, 16); rand.Read(b); token := hex.EncodeToString(b) - Random URL-safe base64 token:
base64.URLEncoding.EncodeToString(b) - Always check the error — entropy can fail in weird containerized environments.
- vs Python: Python has a single
randommodule by default; you have to know to importsecretsfor crypto-grade randomness. Go cleanly separates the two intents at the package level, making the right choice obvious. - vs Java: Java has
Random,ThreadLocalRandom, andSecureRandom— same split, different names. Go's two-package design is clearer. - vs Node.js: Node has
Math.random()(insecure) andcrypto.randomBytes()— same pattern but with the same naming pitfall as Python. - Performance gap:
math/randis ~5–10ns per int.crypto/randis ~100–500ns depending on OS — fast enough for tokens, slow enough that you wouldn't use it for a Monte Carlo simulation.
crypto/rand. If no → math/rand. Mixing them up is one of the most common Go security mistakes: a developer types rand. and uses the first thing autocomplete suggests (math/rand) for a session token. The package names exist specifically to make the right choice unambiguous in code review.- Pre-Go 1.20,
math/randwith no seed always produced the same sequence — code that "worked in tests" was deterministic by accident. - Modulo bias: using
n % 100on a uniformly randomuint64introduces a tiny bias. Userand.Intn(100)which handles this. - Concurrent use:
math/rand's default global is safe for concurrent use but contended; for hot loops, give each goroutine its own*rand.Rand. - Don't roll your own crypto. Even with
crypto/rand, password hashing wantsbcrypt/argon2, not raw SHA-256.
import "math/rand"
// Random integer in range [0, n)
die := rand.Intn(6) + 1 // 1-6 (dice roll)
// Random float in [0.0, 1.0)
f := rand.Float64() // e.g., 0.6046602879796196
// Random number in range [min, max]
min, max := 5, 15
val := rand.Intn(max-min+1) + min // 5 to 15 inclusive
// Seeded random (reproducible) — before Go 1.20
source := rand.NewSource(time.Now().UnixNano())
random := rand.New(source)
target := random.Intn(100) + 1 // 1-100
// Go 1.20+: global rand is auto-seeded, no need for manual seeding
- Jitter in retry/backoff —
time.Sleep(base + rand.Intn(500)*time.Millisecond)avoids thundering herd - Load balancer pick — random server selection for spreading load across N backends
- A/B testing — assign users to variants based on
rand.Float64() < 0.5 - USE
crypto/randfor secrets — passwords, API keys, session IDs —math/randis predictable and unsafe for security - Reproducible tests — seed with a fixed value:
rand.New(rand.NewSource(42))makes tests deterministic - Sampling — picking N items from a slice for log sampling, fuzzing inputs
26 Time, Epoch & Formatting
time package — Go's complete handling of time, dates, durations, timers, and time zones. It centers on three types:
time.Time— a single moment in time, with nanosecond precision and a time zone.time.Duration— a typedint64that counts nanoseconds.5 * time.Second,30 * time.Minute,24 * time.Hour.time.Location— a time zone (e.g.time.UTC,time.Local, or one loaded by name).
"Mon Jan 2 15:04:05 MST 2006" — as the format template. The numbers map to 01/02 03:04:05 PM '06 −0700, which is mnemonic.
- Now:
now := time.Now() - Specific time:
time.Date(2026, time.April, 8, 12, 0, 0, 0, time.UTC) - Parse:
t, err := time.Parse("2006-01-02", "2026-04-08") - Format:
now.Format("2006-01-02 15:04:05") - Add/subtract:
future := now.Add(24 * time.Hour),diff := t2.Sub(t1)(returns Duration). - Compare:
t1.Before(t2),t1.After(t2),t1.Equal(t2)(useEqual, not==!). - Unix epoch:
now.Unix()(seconds),now.UnixMilli(),now.UnixNano(); reverse withtime.Unix(secs, nsec). - Sleep:
time.Sleep(2 * time.Second) - Time zone:
loc, _ := time.LoadLocation("America/New_York"); now.In(loc) - Truncate/Round:
now.Truncate(time.Minute)— drop seconds and below.
01=month, 02=day, 15=hour (24h) or 03=hour (12h), 04=minute, 05=second, 2006=year, MST=timezone, -0700=offset. So:
"2006-01-02"→ ISO date"2006-01-02T15:04:05Z07:00"→ RFC 3339 (the standard Go uses internally)"02/01/2006"→ European format"Mon, 02 Jan 2006 15:04:05 GMT"→ HTTP date format- Pre-defined constants:
time.RFC3339,time.RFC822,time.Kitchen, etc.
- vs Python
strftime: Python uses cryptic codes like%Y-%m-%d %H:%M:%Sthat you have to memorize. Go uses an example date you read literally. - vs Java
SimpleDateFormat: Java'syyyy-MM-dd HH:mm:ssis a different mini-language, andSimpleDateFormatisn't thread-safe (a famous footgun). Go'sFormatis fully thread-safe and the layout is literal. - vs JavaScript
Date: JS'sDateis famously broken (months are 0-indexed, no immutable type, no time zones). Go's API is sound. - Typed durations:
time.Durationis unique — it makes5 * time.Seconda compile-time-checked value. You literally cannot pass a raw int where a duration is expected. - Monotonic clock: Go's
time.Now()includes a monotonic reading, sot2.Sub(t1)works correctly even if the wall clock jumps (NTP, DST).
5 where a time.Duration is expected. The monotonic-clock integration means you don't get negative durations from clock jumps. Time zones are first-class with the IANA database loaded from disk (or embedded with time/tzdata).- Use
t1.Equal(t2), nott1 == t2. The struct comparison includes the location pointer, so two times that represent the same instant in different zones are not==. - Time zone parsing fails on minimal Linux containers without IANA data. Either install
tzdataor import_ "time/tzdata"to embed it. - Don't store times as strings — use
time.Time+ JSON's RFC 3339 marshaling. - UTC for storage, local for display. Always.
Go uses a unique reference time for formatting: Mon Jan 2 15:04:05 MST 2006. This specific date must be used in all format strings — it acts as a template.
Creating & Manipulating Time
import "time"
// Current time
now := time.Now()
fmt.Println(now) // 2024-07-30 12:30:45.123456789 +0530 IST
// Specific time
t := time.Date(2024, time.July, 30, 12, 0, 0, 0, time.UTC)
// Add duration
tomorrow := now.Add(24 * time.Hour)
fmt.Println(tomorrow.Weekday()) // e.g., "Thursday"
// Difference between times
t1 := time.Date(2024, time.July, 4, 12, 0, 0, 0, time.UTC)
t2 := time.Date(2024, time.July, 4, 18, 0, 0, 0, time.UTC)
fmt.Println(t2.Sub(t1)) // 6h0m0s
// Compare times
fmt.Println(t2.After(t1)) // true
fmt.Println(t1.Before(t2)) // true
// Time zones
loc, _ := time.LoadLocation("America/New_York")
nyTime := now.In(loc)
fmt.Println("New York:", nyTime)
// Round and Truncate
fmt.Println(now.Round(time.Hour)) // round to nearest hour
fmt.Println(now.Truncate(time.Hour)) // floor to hour
Time Formatting & Parsing
// Go's reference time: Mon Jan 2 15:04:05 MST 2006
// You must use THESE exact values as placeholders:
// Month: 01 or Jan Day: 02 Year: 2006 or 06
// Hour: 15 (24h) or 03 (12h) Minute: 04 Second: 05
// Format time → string
t := time.Now()
fmt.Println(t.Format("2006-01-02")) // "2024-07-30"
fmt.Println(t.Format("Monday 06-01-02 15-04")) // "Tuesday 24-07-30 14-30"
// Parse string → time
parsed, _ := time.Parse("2006-01-02", "2024-07-30")
fmt.Println(parsed) // 2024-07-30 00:00:00 +0000 UTC
// Parse with time
t1, _ := time.Parse("Jan 02, 2006 03:04 PM", "Jul 03, 2024 03:18 PM")
// ISO 8601 format
t2, _ := time.Parse("2006-01-02T15:04:05Z07:00", "2024-07-04T14:30:18Z")
Epoch / Unix Timestamps
// Unix epoch: seconds since Jan 1, 1970 00:00:00 UTC
now := time.Now()
unixTime := now.Unix() // e.g., 1722345600
fmt.Println("Unix Time:", unixTime)
// Convert Unix timestamp back to time.Time
t := time.Unix(unixTime, 0) // second arg = nanoseconds
fmt.Println(t.Format("2006-01-02")) // "2024-07-30"
// Milliseconds and Nanoseconds
millis := now.UnixMilli() // milliseconds since epoch
nanos := now.UnixNano() // nanoseconds since epoch
- Always use UTC for storage —
time.Now().UTC()before saving to DB; convert to user's timezone only on display - RFC3339 for APIs —
t.Format(time.RFC3339)="2024-07-30T14:30:00Z"— the standard for JSON timestamps - Timeouts everywhere —
context.WithTimeout(ctx, 5*time.Second)for HTTP, DB queries, cache reads - Measuring latency —
start := time.Now(); defer log.Printf("took %v", time.Since(start)) - Cron-like scheduling —
time.NewTickerfor periodic tasks; libraries like robfig/cron for cron expressions - Time mocking in tests — never call
time.Now()directly; inject aClockinterface so tests can advance time deterministically
27 Goroutines & Channels
A goroutine is a lightweight, user-space thread managed by the Go runtime. You start one with the keyword
go placed before any function call: go doWork(arg). The function runs concurrently with the caller; the caller doesn't wait for it.A channel is a typed, thread-safe pipe that lets goroutines send and receive values without explicit locks.
ch <- value sends; v := <-ch receives. Channels are first-class language constructs (not a library), with their own keyword (chan) and operators.Together they implement the famous Go philosophy: "Don't communicate by sharing memory; share memory by communicating."
- Initial stack size: ~2 KB (vs ~1 MB for an OS thread).
- Dynamic stack growth: stacks grow and shrink in segments as needed, up to
GOMAXSTACK(default 1 GB). - M:N scheduler: Go's runtime multiplexes M goroutines onto N OS threads (where N =
GOMAXPROCS, typically the CPU count). - Cooperative + preemptive: goroutines yield at function calls, channel ops, syscalls; since Go 1.14, also preemptively at safe points to prevent CPU-bound goroutines from starving the scheduler.
- You can run millions of goroutines on a single machine — far beyond what OS threads allow.
- Cheap to create: ~10× faster than spawning an OS thread.
- Create:
ch := make(chan int)(unbuffered) ormake(chan int, 100)(buffered, capacity 100). - Send:
ch <- 42— blocks until a receiver is ready (or buffer has space). - Receive:
v := <-ch— blocks until a sender provides a value (or buffer has data). - Close:
close(ch)— signals "no more values"; receivers see zero value +ok=false. - Range:
for v := range ch { ... }— receives until the channel is closed. - Direction: a function param can require send-only (
chan<- T) or receive-only (<-chan T) for safety.
- vs Java threads: Java threads map 1:1 to OS threads (~1 MB each) and use locks/condition variables for coordination. Go goroutines are 500× lighter and channels remove most lock usage.
- vs Node.js: Node has a single-threaded event loop with
async/await. Great for I/O, terrible for CPU work. Go goroutines give you both async I/O and true CPU parallelism with no special syntax. - vs Python
asyncio: Async coroutines but the GIL prevents true parallelism. Plus the entire ecosystem has to be "async-aware" (function coloring problem). Go has no function coloring. - vs C++
std::thread: Heavy OS threads + manual synchronization. Modern C++ is adding coroutines but they're complex. - vs Erlang processes: Closest in spirit — both have lightweight processes and message passing. Go is faster but lacks Erlang's preemption guarantees.
- vs Rust async: Rust has zero-cost async but with steep learning curve (
Pin,Send,Sync, lifetimes). Go is much simpler.
async/await function coloring, no thread-pool tuning, no manual lock juggling. You write straightforward code and the runtime handles parallelism. This is the single feature most Go users cite as the reason they picked the language. Real services routinely run 100,000+ concurrent goroutines.- WaitGroup: wait for N goroutines to finish before continuing.
- Worker pool: N goroutines reading from a shared jobs channel.
- Fan-out: one producer feeds N consumers via the same channel.
- Fan-in: N producers send to one channel; one consumer aggregates.
- Pipeline: stage1 → channel → stage2 → channel → stage3, each stage a goroutine.
- Cancellation via context:
select { case <-ctx.Done(): return; case ... }
- Goroutine leaks: a goroutine blocked on a channel that never receives runs forever. Always have a way out — context, timeout, or close.
- Loop variable capture (pre-1.22):
for _, v := range s { go f(v) }— all goroutines saw the samev. Fixed in Go 1.22. - Sending to a closed channel panics. Receiving from one returns zero value + ok=false.
- Don't close a channel from the receiver side — only the sender knows when it's done.
Goroutines
// Lightweight thread (~2KB stack vs ~1MB OS thread)
go doWork(1) // launch goroutine
// WaitGroup — wait for goroutines to finish
var wg sync.WaitGroup // counter: Add/Done/Wait
for i := 0; i < 5; i++ {
wg.Add(1) // increment counter before goroutine
go func(id int) { // id passed to avoid capture bug
defer wg.Done() // decrement when goroutine exits
doWork(id)
}(i) // pass i as argument immediately
}
wg.Wait() // block until counter hits zero
// BUG: loop variable captured by reference
for i := 0; i < 5; i++ {
go func() { fmt.Println(i) }() // prints 5,5,5,5,5
}
// FIX: pass as argument
for i := 0; i < 5; i++ {
go func(n int) { fmt.Println(n) }(i) // copy i into n
}
// Go 1.22+: loop vars are per-iteration (fixed!)
Channels
// Unbuffered — sender blocks until receiver ready
ch := make(chan int) // synchronous handoff
go func() { ch <- 42 }() // goroutine sends 42
val := <-ch // 42
// Buffered — blocks only when buffer full
bch := make(chan string, 3) // can send 3 items without waiting
// Direction constraints
func producer(ch chan<- int) { ch <- 1 } // send-only
func consumer(ch <-chan int) { <-ch } // receive-only
// Range over channel (until closed)
for v := range ch { fmt.Println(v) } // exits when ch is closed
// Worker pool pattern
jobs := make(chan int, 100) // input queue
results := make(chan int, 100) // output queue
for w := 0; w < 3; w++ { // 3 worker goroutines
go func() {
for job := range jobs { // read until jobs closed
results <- job * 2 // process and send result
}
}()
}
_ = val; _ = bch // suppress unused-var errors
- HTTP servers — Go's
net/httpspawns one goroutine per request automatically; thousands run concurrently with minimal overhead - Background jobs — email sending, image processing, webhook delivery — fire-and-forget with logging
- Fan-out / fan-in — process N items in parallel, collect results — used in batch processors, web scrapers
- Periodic health checks —
go func() { for { check(); time.Sleep(30*time.Second) } }() - Always use WaitGroup or channel — never start a goroutine and forget about it (leaks memory + makes shutdown impossible)
- Pass values, not loop vars —
go func(id int) { ... }(i)avoids the classic capture bug (fixed in Go 1.22+)
28 Channel Deep Dive
go once" from "I write production Go services". The concepts:
- Unbuffered channels — sender and receiver rendezvous: both block until both are ready.
- Buffered channels — capacity > 0; send blocks only when full, receive blocks only when empty.
- Directional channels —
chan<- T(send-only) and<-chan T(receive-only) for type-safe API contracts. - Closing channels —
close(ch)signals "no more values"; receivers detect with the comma-ok idiom. - Channel as a synchronization primitive — using sends/receives as signals, not just data transfer.
- Nil channels — sending to or receiving from a nil channel blocks forever; useful in
selectto disable cases.
- Unbuffered (
make(chan int)): A pure handoff. Sender blocks until a receiver is ready; receiver blocks until a sender provides. Guarantees happens-before: the send completes before the receive returns. Use when you need synchronization or a "bounded queue of size 1 in flight". - Buffered (
make(chan int, n)): Acts like a bounded FIFO queue. Sends only block when the buffer is full; receives only block when it's empty. Use when producer and consumer have different rates and you want to absorb bursts. - Choosing capacity: 0 = synchronous handoff. Small (1–10) = decouple producer/consumer slightly. Large = pure queue (be careful — large buffers hide latency problems).
- Only the sender should close. Never close from the receiver side — you might race the sender and panic.
- Sending on a closed channel panics — irrecoverable.
- Receiving from a closed channel returns the zero value and
ok = false:v, ok := <-ch. - Range loops (
for v := range ch) automatically exit when the channel is closed and drained. - Closing a nil channel panics.
- Closing twice panics.
- You don't have to close every channel. Garbage collection will clean up channels with no remaining references. Close only to signal "no more data" to receivers.
func produce(out chan<- int)— function may only send toout; trying to read is a compile error.func consume(in <-chan int)— function may only receive fromin; trying to send is a compile error.- A bidirectional
chan Tcan be implicitly converted to either direction; the reverse is not allowed. - This is your most powerful API documentation tool — the type system enforces the protocol.
- vs queues in libraries (Java
BlockingQueue, Pythonqueue.Queue): Those are heap objects with method-call overhead. Go channels are part of the runtime — the scheduler can park/wake goroutines almost for free when they block. - Direction typing is unique — no mainstream queue library lets you statically prevent a producer from reading.
- vs Erlang messages: Erlang has typeless mailboxes per process. Go channels are typed pipes between any two goroutines, more flexible but less actor-like.
- vs Rust mpsc: Similar in spirit but Rust splits sender and receiver into two types; Go uses one channel value with direction-cast on function args.
select cases — these are the details that prevent the deadlocks and goroutine leaks that bite beginners. Mastering channel deep mechanics is the difference between writing toy concurrent code and building robust production services.- Done channel:
done := make(chan struct{}); <-doneas a wait-forever signal;close(done)to release everyone. - Quit channel: close-as-broadcast cancellation.
- Semaphore: buffered channel of size N as a counting semaphore —
sem <- struct{}{}to acquire,<-semto release. - Barrier: close an unbuffered channel after all workers have signaled.
- Generator: a goroutine that pushes values onto a channel until exhausted, then closes.
Unbuffered vs Buffered Channels
// UNBUFFERED — synchronous handoff
// Sender blocks UNTIL receiver is ready (and vice versa)
ch := make(chan int) // no capacity = unbuffered
go func() {
time.Sleep(3 * time.Second)
fmt.Println(<-ch) // receives after 3 seconds
}()
ch <- 1 // BLOCKS here for 3 seconds until receiver is ready
fmt.Println("End of program")
// BUFFERED — async up to capacity
// Blocks on send ONLY when buffer is FULL
// Blocks on receive ONLY when buffer is EMPTY
bch := make(chan int, 2) // capacity 2
bch <- 1 // doesn't block (buffer has space)
bch <- 2 // doesn't block (buffer has space)
// bch <- 3 // BLOCKS! buffer is full, needs a receiver
go func() {
time.Sleep(2 * time.Second)
fmt.Println("Received:", <-bch) // frees one slot
}()
bch <- 3 // now this can send (after goroutine receives)
Channel Directions
// Restrict a channel to send-only or receive-only in function signatures
// This catches bugs at compile time!
func producer(ch chan<- int) { // chan<- = SEND-ONLY
for i := range 5 {
ch <- i
}
close(ch) // producer closes the channel when done
}
func consumer(ch <-chan int) { // <-chan = RECEIVE-ONLY
for value := range ch {
fmt.Println("Received:", value)
}
}
ch := make(chan int)
go producer(ch) // auto-converts to chan<-
consumer(ch) // auto-converts to <-chan
Closing Channels
// Only the SENDER should close a channel, never the receiver
// Sending to a closed channel = PANIC
// Receiving from a closed channel = returns zero value immediately
ch := make(chan int)
// Check if channel is closed using comma-ok idiom
close(ch)
val, ok := <-ch
if !ok {
fmt.Println("Channel is closed") // ok=false means closed
}
// Pipeline pattern: producer → filter → consumer
func producer(ch chan<- int) {
for i := range 5 { ch <- i }
close(ch)
}
func filter(in <-chan int, out chan<- int) {
for val := range in {
if val%2 == 0 { out <- val } // only pass even numbers
}
close(out)
}
ch1 := make(chan int)
ch2 := make(chan int)
go producer(ch1)
go filter(ch1, ch2)
for val := range ch2 { fmt.Println(val) } // 0, 2, 4
Non-Blocking Channel Operations
// Use select + default for non-blocking send/receive
ch := make(chan int)
// Non-blocking receive
select {
case msg := <-ch:
fmt.Println("Received:", msg)
default:
fmt.Println("No messages available.") // runs immediately
}
// Non-blocking send
select {
case ch <- 1:
fmt.Println("Sent message.")
default:
fmt.Println("Channel not ready.") // runs if no receiver
}
// Real-time: poll for data with quit signal
data := make(chan int)
quit := make(chan bool)
go func() {
for {
select {
case d := <-data:
fmt.Println("Data:", d)
case <-quit:
fmt.Println("Stopping..."); return
default:
fmt.Println("Waiting...")
time.Sleep(500 * time.Millisecond)
}
}
}()
Channel Synchronization
// Use channels to synchronize goroutine completion
// Simple done signal
done := make(chan int)
go func() {
fmt.Println("Working...")
time.Sleep(2 * time.Second)
done <- 0 // signal completion
}()
<-done // block until goroutine is done
// Wait for multiple goroutines
numGoroutines := 3
done = make(chan int, numGoroutines)
for i := range numGoroutines {
go func(id int) {
fmt.Printf("Goroutine %d working...\n", id)
time.Sleep(time.Second)
done <- id // signal completion
}(i)
}
for range numGoroutines {
<-done // wait for each one
}
fmt.Println("All goroutines complete")
- Pipeline architecture — stream stages: parse → validate → enrich → write, each stage in its own goroutine, connected by channels
- Bounded queue — buffered channel as a job queue:
make(chan Job, 1000)drops/blocks if full - Cancellation signal — close a
chan struct{}to broadcast "stop" to all listeners (zero-byte messages) - Result aggregation — N workers send results into one channel; main goroutine ranges over it
- Don't share channels across packages carelessly — channel ownership (who closes it) should be obvious
- chan struct{} for signals — used over
chan boolwhen you only need notification, not data
29 Select & Multiplexing
switch, but each case is a channel send or receive instead of a value comparison. The first case whose channel operation can proceed runs. If multiple are ready, one is chosen at random (to prevent starvation).Special cases:
defaultcase — runs immediately if no other case is ready, makingselectnon-blocking.- No cases ready, no default — the goroutine blocks until any case becomes ready.
- Empty
select {}— blocks forever, occasionally used to keep the main goroutine alive. - Nil channels in cases — never ready, effectively disabling that case.
- Multi-way receive: wait for whichever channel produces first.
- Receive with timeout:
case <-time.After(5 * time.Second): return ErrTimeout - Non-blocking receive:
select { case v := <-ch: ...; default: ... }— try to read, fall back if nothing's there. - Non-blocking send:
select { case ch <- v: ...; default: ... }— try to send, drop the message if nobody's listening. - Cancellation watch:
case <-ctx.Done(): return ctx.Err()— the canonical Go cancellation idiom. - Heartbeat:
case <-ticker.C: sendHeartbeat()alongside the work case.
- vs
switch: syntactically similar, semantically very different. Switch compares values; select waits on channel operations. - vs Unix
select(2)/epoll: Same idea (wait on many I/O operations) but at the language level for arbitrary typed channels, not file descriptors. - vs Java
Selector/BlockingQueue.poll: Java requires manual loops and explicit polling. Go'sselectis one expression. - vs Node.js / Python asyncio: Other languages need
Promise.raceorasyncio.waitwith callback chains. Go has a single blocking, multi-way primitive built into the syntax. - vs Rust
tokio::select!: Rust now has an analogous macro, inspired in part by Go. - Random selection when multiple cases are ready is unique — prevents one channel from monopolizing if it's always ready.
select is the building block for timeouts, cancellation, fan-in/fan-out, heartbeats, retries, rate limiting, and graceful shutdown. Pretty much every long-running goroutine in production Go has a select watching at minimum ctx.Done() and a work channel. Without select, you'd be reduced to busy-polling or one-channel-at-a-time blocking — both terrible.- Cancellable receive:
select { case v := <-work: handle(v); case <-ctx.Done(): return } - Timeout wrapper:
select { case r := <-resp: return r; case <-time.After(timeout): return ErrTimeout } - Drop on overflow: non-blocking send to a buffered channel; if full, drop the metric/log and increment a counter.
- Throttle: wait on a ticker channel before each operation.
- Disable a case dynamically: set the channel variable to nil — that case becomes unreachable.
- time.After leaks in long loops: each call allocates a new timer that lives until it fires. In tight loops, prefer
time.NewTimer+Reset. - The order of cases in source doesn't matter — selection is random when multiple are ready.
- Nested selects can be confusing; consider extracting helper functions.
- Receiving from a closed channel always succeeds immediately — useful as a broadcast cancel signal, but means a closed channel "wins" every selection.
// select — wait on multiple channel operations
select { // blocks until one case is ready
case msg := <-ch1: // receive from ch1
fmt.Println(msg)
case msg := <-ch2: // receive from ch2
fmt.Println(msg)
case <-time.After(1 * time.Second): // fires after 1s
fmt.Println("timeout!")
default: // non-blocking: runs if nothing else ready
fmt.Println("no message ready")
}
// Ticker — periodic events
ticker := time.NewTicker(500 * time.Millisecond) // fires every 500ms
defer ticker.Stop() // clean up goroutine when done
for {
select {
case <-done: return // exit on done signal
case t := <-ticker.C: fmt.Println("tick", t) // handle each tick
}
}
- Timeouts on operations —
case <-time.After(5*time.Second): return ErrTimeout— most common Go pattern - Multiplexing channels — combining job queue + cancellation + heartbeat in one event loop
- Fair channel reads —
selectchooses randomly among ready cases, preventing starvation - Graceful shutdown loops —
case <-ctx.Done(): return ctx.Err()in worker loops - Non-blocking try-send —
defaultbranch lets you drop messages instead of blocking when consumers are slow - Server main loop — Kubernetes controllers, queue consumers all run on one big
for { select { ... } }
30 Worker Pools
jobs channel and (optionally) write outcomes to a results channel. The pool size N caps how much work happens in parallel — providing automatic backpressure and bounded resource usage. Closing the jobs channel signals every worker to drain remaining work and exit; a sync.WaitGroup coordinates the final shutdown.- Jobs channel: producer-to-worker conduit, typically
chan Job. - Results channel: worker-to-collector conduit, typically
chan Result. Buffered to match concurrency. - N worker goroutines: each runs
for job := range jobs { results <- process(job) }. - WaitGroup: tracks workers so the collector knows when to close
results. - Closer goroutine: after
wg.Wait(), callsclose(results)so the consumer's range loop terminates. - Optional context: for cancellation; workers select between
jobsandctx.Done().
- vs Java
ExecutorService/ThreadPoolExecutor: Java's pools are heavy library types with config classes, factory methods, and shutdown protocols. A Go worker pool is ~20 lines of plain code using just goroutines, channels, andWaitGroup. The pattern is the framework. - vs Python
concurrent.futures: Same idea but Python's GIL means CPU-bound work doesn't actually parallelize. Go workers run on real OS threads. - vs Node.js
worker_threads: Heavy and rarely used; Node prefers event loop. Go pools combine the best of both. - vs unbounded "goroutine per request": Spawning a goroutine per HTTP request seems convenient but can blow up under load — exhausting file descriptors, overloading downstreams, or triggering OOM. A pool gives you a natural admission gate.
- CPU-bound work: N =
runtime.NumCPU()or slightly less. - I/O-bound work: N can be much higher — 50 to 500 — since workers spend most time waiting.
- Mixed: measure with
pprofand a load test; the right number is rarely guessable. - Per-resource limits: if each worker holds a DB connection, N ≤ pool size − safety margin.
- Errgroup pattern: use
golang.org/x/sync/errgroup— combines WaitGroup, error capture, and context cancellation in one type. - Bounded semaphore: a buffered channel of size N as a counting semaphore — even simpler than a pool when you don't need long-lived workers.
- Pipelined pools: several pools connected in stages, each transforming output of the previous.
- Pull vs push: workers pulling from a channel scales naturally; pushing to specific workers requires routing logic.
Worker pools limit concurrency by spinning up a fixed number of goroutines that read from a shared jobs channel. This prevents spawning unlimited goroutines and controls resource usage.
// Basic worker pool pattern
func worker(id int, tasks <-chan int, results chan<- int) {
for task := range tasks {
fmt.Printf("Worker %d processing task %d\n", id, task)
time.Sleep(time.Second) // simulate work
results <- task * 2 // send result
}
}
func main() {
numWorkers := 3
numJobs := 10
tasks := make(chan int, numJobs)
results := make(chan int, numJobs)
// Start 3 workers — they all read from same channel
for i := range numWorkers {
go worker(i, tasks, results)
}
// Send 10 tasks
for i := range numJobs { tasks <- i }
close(tasks) // signal no more tasks
// Collect all results
for range numJobs {
fmt.Println("Result:", <-results)
}
}
Real-World Example: Ticket Processor
type ticketRequest struct {
personID int
numTickets int
cost int
}
func ticketProcessor(requests <-chan ticketRequest, results chan<- int) {
for req := range requests {
fmt.Printf("Processing %d ticket(s) for person %d, cost $%d\n",
req.numTickets, req.personID, req.cost)
time.Sleep(time.Second) // simulate processing
results <- req.personID
}
}
// 3 workers process 5 ticket requests concurrently
ticketRequests := make(chan ticketRequest, 5)
ticketResults := make(chan int)
for range 3 { go ticketProcessor(ticketRequests, ticketResults) }
- Image/video processing — fixed pool of workers thumbnails uploaded files; protects CPU/memory limits
- Rate-limited API calls — N workers = max N concurrent calls to a third-party API (e.g., 10 to OpenAI)
- DB write batching — bounded number of DB connections; jobs queue waits when all workers are busy
- Web crawlers — limit concurrent HTTP fetches to avoid being IP-banned
- Email/notification senders — process messages from a queue (Redis/Kafka) with controlled parallelism
- errgroup.SetLimit — Go's
x/sync/errgrouphas built-in limits; often simpler than manual worker pools
31 Timers & Tickers
time package, both built around channels:
time.Timer— fires once after a specified duration. Created withtime.NewTimer(d)ortime.AfterFunc(d, fn). Exposes a channelCon which it sends the current time when it fires.time.Ticker— fires repeatedly at a fixed interval. Created withtime.NewTicker(d). Exposes a channelCthat periodically delivers the current time.
time.Sleep(d) blocks the goroutine for d; time.After(d) returns a one-shot receive-only channel that fires after d (perfect for inline use in a select); time.AfterFunc(d, fn) runs fn in its own goroutine after d.
- Create timer:
t := time.NewTimer(5 * time.Second) - Wait for fire:
<-t.C - Stop early:
t.Stop()— returns true if successfully stopped before firing. - Reset:
t.Reset(d)— reuse the timer with a new duration. - Create ticker:
tk := time.NewTicker(1 * time.Second) - Read tick:
<-tk.Cin a loop orselect. - ALWAYS stop tickers:
defer tk.Stop()— otherwise the runtime keeps sending forever. - One-shot inline:
case <-time.After(2 * time.Second): // timeout
- vs JavaScript
setTimeout/setInterval: JS uses callbacks; Go uses channels. Go's channel-based approach composes withselectfor free — same primitive for timers and any other concurrent operation. - vs Java
ScheduledExecutorService: Java requires a thread pool and scheduling primitives. Go uses the runtime's built-in heap of timers — much lighter. - vs Python
threading.Timer/asyncio.sleep: Python's are callback or coroutine-based. Go's are values you pass around. - vs Unix
setitimer: OS-level signals are awkward and process-wide. Go timers are per-goroutine and compose freely.
select alongside your work channel and ctx.Done() for elegant timeout-and-cancel handling.- Tickers leak if not stopped. Always
defer tk.Stop()immediately after creation — forgotten tickers keep firing forever, slowly burning CPU. time.Afterin long-running loops leaks memory — each iteration allocates a new timer that lives until it fires. Usetime.NewTimer+Resetin hot loops.- Ticker drops ticks under load. If your handler is slow, you get the next tick whenever you're ready; ticks aren't queued. This is by design — prevents runaway pile-ups.
- Reset on a fired timer needs draining the channel first, otherwise you'll get the old value. Pre-Go 1.23 this was a famous footgun; modern Go fixed the semantics.
- For testability, abstract
time.Now/time.NewTickerbehind an interface so tests can use a fake clock (benbjohnson/clock).
Timers (fire once)
// Timer fires ONCE after a duration
timer := time.NewTimer(2 * time.Second)
<-timer.C // blocks until timer fires (timer.C is a channel)
fmt.Println("Timer expired")
// Stop a timer before it fires
timer2 := time.NewTimer(5 * time.Second)
stopped := timer2.Stop() // returns true if successfully stopped
if stopped { fmt.Println("Timer stopped before firing") }
// Reset a timer
timer2.Reset(time.Second) // restart with new duration
// Timeout pattern with select
timeout := time.After(3 * time.Second) // returns <-chan time.Time
done := make(chan bool)
go func() { longRunningTask(); done <- true }()
select {
case <-timeout:
fmt.Println("Operation timed out!")
case <-done:
fmt.Println("Operation completed")
}
// Racing two timers
t1 := time.NewTimer(1 * time.Second)
t2 := time.NewTimer(2 * time.Second)
select {
case <-t1.C: fmt.Println("Timer1 won") // fires first
case <-t2.C: fmt.Println("Timer2 won")
}
Tickers (fire repeatedly)
// Ticker fires REPEATEDLY at intervals
ticker := time.NewTicker(time.Second)
defer ticker.Stop() // ALWAYS stop tickers to free resources
stop := time.After(5 * time.Second)
for {
select {
case tick := <-ticker.C:
fmt.Println("Tick at:", tick) // fires every 1s
case <-stop:
fmt.Println("Stopping ticker."); return
}
}
// Use case: periodic health checks, polling, heartbeats
func periodicTask() { fmt.Println("Health check at:", time.Now()) }
ticker := time.NewTicker(2 * time.Second)
defer ticker.Stop()
for range ticker.C { periodicTask() } // runs forever
- Cache TTL —
time.AfterFunc(5*time.Minute, func() { delete(cache, key) })for auto-expiring entries - Heartbeat to load balancer — ticker fires every N seconds to send "I'm alive" to a discovery service (Consul, etcd)
- Metrics flush — Prometheus scrape interval, StatsD flush — all driven by tickers
- Batch processor — flush every N seconds OR when batch size reaches K — combines ticker + buffer
- ALWAYS
defer ticker.Stop()— leaked tickers fire forever and prevent garbage collection - time.After in long-running loops — leaks memory each iteration; prefer
time.NewTimer+Reset
32 Sync Package
sync package — Go's collection of classic shared-memory concurrency primitives. Although Go's slogan is "don't communicate by sharing memory; share memory by communicating", these primitives exist for the cases where channels are awkward or overkill. The package provides:
Mutex— mutual exclusion lock; one goroutine at a time.RWMutex— readers/writer lock; many readers OR one writer.WaitGroup— wait for N goroutines to finish.Once— guarantee a function runs exactly once across all goroutines.Cond— condition variable for signal/wait coordination.Map— concurrent map optimized for specific access patterns.Pool— per-P object recycler to reduce GC pressure.OnceFunc/OnceValue/OnceValues(Go 1.21+) — convenient lazy-initialization wrappers.
sync.Mutex:var mu sync.Mutex; mu.Lock(); defer mu.Unlock(); /* critical section */. The zero value is an unlocked mutex — no constructor needed.sync.RWMutex:mu.RLock()for readers,mu.Lock()for writers. Pays off when reads vastly outnumber writes.sync.WaitGroup:wg.Add(1)before each goroutine,wg.Done()at end of each,wg.Wait()in the parent.sync.Once:once.Do(func() { /* runs exactly once */ })— the standard idiom for lazy singleton initialization.
- Zero value usability: every
synctype works straight fromvar x sync.Mutex— no constructors, nonilchecks. - vs Java
synchronizedkeyword: Go locks are explicit values, not keywords on methods. More flexible (lock can be embedded, passed) but require discipline. - vs C++
std::mutex: Very similar shape, but Go lacks C++'s RAII guards (std::lock_guard) — usedefer Unlock()instead. - vs Python
threading: Python's GIL means true parallelism is rare; Go's primitives matter more because real concurrency happens. sync.Onceis unique — most languages need an "atomic boolean flag + double-checked locking" dance. Go gives you the right answer in one method call.
Mutex is simpler and faster than a goroutine + channel doing the same job. sync.Once is the cleanest way to lazily initialize a singleton (DB pool, config, regex). sync.Pool dramatically reduces allocations in hot paths (used by fmt, encoding/json, net/http internally). Knowing both styles — channels and locks — and when to pick which is what makes Go concurrency feel natural.- Always pair Lock/Unlock with defer immediately after Lock — guarantees release on panic or early return.
- Don't copy a Mutex after first use;
go vetcatches this. Always use pointer receivers on types containing mutexes. - Lock as late as possible, unlock as early as possible — minimize the critical section.
- Choose RWMutex only with measurement — its overhead exceeds Mutex unless reads vastly outnumber writes.
- Don't mix channel and mutex synchronization on the same data — pick one model per piece of state.
- Don't reset a WaitGroup while goroutines are still using it — recreate it instead.
// Mutex
type SafeCounter struct {
mu sync.Mutex // protects v from concurrent access
v map[string]int // shared mutable state
}
func (c *SafeCounter) Inc(key string) {
c.mu.Lock() // acquire exclusive lock
defer c.mu.Unlock() // always release lock
c.v[key]++ // safe to modify now
}
// RWMutex — multiple readers OR one writer
type Cache struct { mu sync.RWMutex; data map[string]string }
func (c *Cache) Get(k string) string {
c.mu.RLock(); defer c.mu.RUnlock() // read lock allows concurrent reads
return c.data[k]
}
// sync.Once — run exactly once (singleton)
var once sync.Once // goroutine-safe one-time init
once.Do(func() { /* expensive init */ }) // only runs once ever
// sync.Pool — reuse objects
// sync.Map — concurrent map
// atomic — lock-free operations
var counter int64 // shared counter
atomic.AddInt64(&counter, 1) // increment atomically, no mutex needed
- sync.Mutex around shared maps — every in-memory cache, session store, rate limiter
- sync.RWMutex for read-heavy — config reload (rare write, many reads), feature flags, route tables
- sync.Once for lazy init — expensive global init (DB connection pool, parsing templates) deferred until first use
- sync.WaitGroup for fan-out — wait for N parallel HTTP fetches before aggregating results
- defer Unlock immediately after Lock — guarantees release even on panic; standard Go style
- Don't copy a Mutex — vet warns about this; always use pointer receivers on types containing mutexes
33 Sync Deep Dive
sync package — the ones you reach for when a plain Mutex isn't enough or when profiling reveals contention or allocation pressure. These are specialized tools, each with niche use cases and easy-to-make mistakes.- API:
RLock/RUnlockfor readers,Lock/Unlockfor writers. - Behavior: any number of readers can hold the lock simultaneously, but a writer requires exclusive access.
- When it pays off: only when reads vastly outnumber writes (typically 100:1 or more) AND the critical section is meaningful work. For trivial sections, plain
Mutexis faster. - Writer starvation: a continuous stream of readers can starve writers — Go's implementation handles this but with some latency cost.
- API:
Store,Load,LoadOrStore,LoadAndDelete,Delete,Range. - Optimized for two patterns: (1) keys written once, read many times (caches); (2) goroutines accessing disjoint key sets.
- NOT a drop-in for
map+Mutex. For write-heavy workloads, the regular map + lock is significantly faster. - No len() and no type safety: uses
anyfor key and value, requiring assertions. - Range can see stale or in-flight values — not a snapshot.
- Purpose: reuse short-lived allocations to reduce GC pressure.
- API:
p := sync.Pool{New: func() any { return new(Buffer) }};buf := p.Get().(*Buffer); /* use */; p.Put(buf). - Per-P sharding: internally has one pool per scheduler P, so contention is minimal.
- GC may evict items at any time. Unlike traditional pools,
Poolis a cache, not a guarantee — items can vanish between Put and Get. - Used in stdlib:
fmt,encoding/json,net/http,bytes.Bufferinternals. - Always reset Pool items before Put — they may carry stale data from previous use.
- Pattern: hold a lock, wait for a condition to become true, get notified by another goroutine.
- API:
c := sync.NewCond(&mu); goroutines callc.Wait()(releases lock, waits, re-acquires); signaler callsc.Signal()(wake one) orc.Broadcast()(wake all). - Almost always overkill in Go — channels handle most signal/wait scenarios more cleanly. The Go team has openly discussed deprecating
Cond.
- Standard idiom:
var once sync.Once; var db *DB; func GetDB() *DB { once.Do(func() { db = connect() }); return db } - The function inside
Doruns exactly once across all goroutines, even under contention. - Modern alternatives (Go 1.21+):
sync.OnceFunc,sync.OnceValue,sync.OnceValueswrap the pattern in a function returning closures.
RWMutex), allocation pressure (consider Pool), or singleton init complexity (use Once). Knowing the niches keeps you from misapplying them — and equally important, from not using them when they'd help.RWMutex — Multiple Readers OR One Writer
Unlike Mutex, RWMutex allows concurrent reads but exclusive writes. Use when reads far outnumber writes (e.g., caches, config).
var (
rwmu sync.RWMutex
counter int
)
func readCounter(wg *sync.WaitGroup) {
defer wg.Done()
rwmu.RLock() // multiple goroutines can RLock simultaneously
fmt.Println("Read:", counter)
rwmu.RUnlock()
}
func writeCounter(wg *sync.WaitGroup, value int) {
defer wg.Done()
rwmu.Lock() // exclusive — blocks ALL readers and writers
counter = value
fmt.Printf("Written: %d\n", value)
rwmu.Unlock()
}
// 5 concurrent readers, then 1 writer, all safe
sync.Once — Run Exactly Once
// sync.Once guarantees a function runs only ONCE, even across goroutines
// Perfect for: singleton initialization, one-time setup
var once sync.Once
func initialize() {
fmt.Println("This runs only ONCE no matter how many goroutines call it")
}
var wg sync.WaitGroup
for i := range 5 {
wg.Add(1)
go func() {
defer wg.Done()
fmt.Println("Goroutine #", i)
once.Do(initialize) // only first goroutine runs initialize()
}()
}
wg.Wait()
sync.Cond — Condition Variables
sync.Cond allows goroutines to wait for a condition and be woken up by another goroutine. Classic use: producer-consumer with bounded buffer.
const bufferSize = 5
type buffer struct {
items []int
mu sync.Mutex
cond *sync.Cond
}
func newBuffer() *buffer {
b := &buffer{items: make([]int, 0, bufferSize)}
b.cond = sync.NewCond(&b.mu) // cond is linked to the mutex
return b
}
func (b *buffer) produce(item int) {
b.mu.Lock()
defer b.mu.Unlock()
for len(b.items) == bufferSize {
b.cond.Wait() // wait until consumer frees space
}
b.items = append(b.items, item)
fmt.Println("Produced:", item)
b.cond.Signal() // wake up one waiting goroutine
}
func (b *buffer) consume() int {
b.mu.Lock()
defer b.mu.Unlock()
for len(b.items) == 0 {
b.cond.Wait() // wait until producer adds items
}
item := b.items[0]
b.items = b.items[1:]
fmt.Println("Consumed:", item)
b.cond.Signal() // wake up producer if it was waiting
return item
}
What problem does it solve?
A Mutex protects shared data, but it can't tell you when a condition becomes true. Without Cond, a goroutine that needs to wait for "buffer not full" or "queue has items" would either:
- Spin in a loop — keep polling
len(buf) < maxwhile wasting CPU - Sleep for arbitrary durations — adds latency, still wastes cycles, races with state changes
sync.Cond lets a goroutine sleep efficiently until another goroutine explicitly wakes it up — no polling, no spinning.
The three operations:
| Method | What it does |
|---|---|
cond.Wait() | Atomically unlocks the mutex AND blocks the caller. When woken up, it re-locks the mutex before returning. You MUST hold the lock before calling Wait. |
cond.Signal() | Wakes up one waiting goroutine (if any). Caller can hold the lock or not — but typically does. |
cond.Broadcast() | Wakes up all waiting goroutines. Use when the condition change can satisfy more than one waiter. |
Why Wait() must be called inside a for loop, not if:
// CORRECT — re-checks condition after wake-up
for len(b.items) == bufferSize {
b.cond.Wait() // wakes up holding lock
}
// WRONG — assumes condition is true after Wait returns
if len(b.items) == bufferSize {
b.cond.Wait()
}
Why? Between the time Wait wakes up and re-acquires the lock, another goroutine may have already taken the slot. Also, Go's runtime is allowed to perform spurious wakeups. The for loop guarantees the condition is actually true before proceeding.
The atomicity trick — what makes Cond special:
The line cond.Wait() does three things atomically:
- Releases the mutex
- Suspends the goroutine on a wait queue
- (Later, when signaled) re-acquires the mutex before returning
This atomicity is critical. If steps 1 and 2 weren't atomic, a producer could call Signal() in the gap between "release mutex" and "go to sleep" — the consumer would miss the signal and sleep forever (a "lost wake-up").
Signal vs Broadcast — when to use which:
- Signal — when only one waiter can make progress per state change (e.g., one slot freed in a buffer = one producer can write)
- Broadcast — when the change can unblock multiple waiters (e.g., a "shutdown" flag set, all consumers should wake and check it; or "data is now ready" for many readers)
Producer/consumer flow visualized:
Consumer goroutine: Producer goroutine:
───────────────────── ─────────────────────
mu.Lock() mu.Lock()
for len(items) == 0 { items = append(items, x)
cond.Wait() ◄─────signal───── cond.Signal()
} mu.Unlock()
item := items[0]
items = items[1:]
mu.Unlock()
When to use Cond vs alternatives:
- Use
Cond— when goroutines need to wait for a complex predicate on shared state (multi-condition queues, buffer pools with min/max thresholds) - Use a channel instead — for simple signaling or producer/consumer; channels are idiomatic Go and easier to reason about.
chan Tis essentially a built-in Cond + buffer + lock - Use
sync.WaitGroup— when you just need to wait for N goroutines to finish
Why Cond is rare in idiomatic Go:
Most condition-waiting needs are better served by channels. Cond is primarily used in performance-critical libraries (e.g., the runtime itself, sync.Pool, semaphore implementations) where channel overhead matters or the predicate is too complex to express as a channel pattern.
Common pitfalls:
- Calling
Wait()without holding the lock → panic - Using
ifinstead offoraroundWait()→ race condition / spurious wakeup bug - Calling
Signal()when no one is waiting → silently does nothing (signals are not queued!) - Forgetting that
Condis not safe to copy after first use (always use a pointer)
sync.Pool — Object Reuse
sync.Pool caches allocated objects for reuse, reducing GC pressure. Objects may be garbage collected at any time — don't store things you need to keep.
type person struct { name string; age int }
var pool = sync.Pool{
New: func() interface{} {
fmt.Println("Creating a new Person.")
return &person{} // factory for new objects
},
}
// Put an object into the pool
pool.Put(&person{name: "John", age: 81})
// Get reuses an existing object (or creates new via New func)
p1 := pool.Get().(*person)
fmt.Println(p1) // &{John 81}
// Return to pool when done
pool.Put(p1)
// If pool is empty and no New func: Get() returns nil
p2 := pool.Get()
if p2 != nil {
fmt.Println("Reused:", p2)
} else {
fmt.Println("Pool is empty")
}
- RWMutex — config service hot-reloads from disk, thousands of read requests share the lock; one writer briefly takes exclusive access
- sync.Once — lazy DB connection:
dbOnce.Do(func(){ db = connect() })inGetDB()— first call connects, all others get the cached pool - sync.Cond — bounded buffer pools, custom semaphores, "wait until N items available" predicates
- sync.Pool — JSON encoders reuse
bytes.Bufferobjects;fmtpackage internally uses Pool for printer state — saves millions of allocations under load - sync.Map — read-mostly maps where keys are added once and read many times (e.g., cache of compiled regexes by pattern)
- Don't use sync.Map by default — regular map + Mutex is faster for write-heavy workloads
34 Atomic Operations
sync/atomic package — lock-free operations on integers and pointers, implemented via special CPU instructions. The traditional API uses free functions (atomic.AddInt64, atomic.LoadUint32, etc.) that take *T arguments. Go 1.19+ introduced typed wrappers (atomic.Int64, atomic.Uint32, atomic.Bool, atomic.Pointer[T], atomic.Value) that are method-based and harder to misuse.int32, int64, uint32, uint64, uintptr, unsafe.Pointer):
Load— atomic read.Store— atomic write.Add— atomic add (returns the new value).Swap— atomic exchange (returns the old value).CompareAndSwap(CAS) — atomic "if value == old, replace with new" (returns success bool). The foundation of lock-free data structures.And/Or(Go 1.23+) — atomic bitwise operations.
var counter atomic.Int64 counter.Add(1) v := counter.Load() counter.Store(0) ok := counter.CompareAndSwap(0, 100)The typed wrappers prevent the most common bug with the old API: forgetting that the variable must be aligned. They also prevent accidentally mixing atomic and non-atomic access (which would race).
- vs Mutex: A
Mutexcan block contending goroutines and involves the scheduler. Atomics use single CPU instructions (CMPXCHG, LOCK ADD, etc.) that complete in a few nanoseconds with no kernel involvement. - One value at a time: atomics work on a single machine word. Updating two related fields atomically still requires a mutex (or CAS on a struct pointer).
- vs Java
AtomicInteger: nearly identical API and use cases. Both wrap the same underlying CPU instructions. - vs C++
std::atomic: Go's atomics are sequentially consistent by default; C++ exposes weaker memory orderings (acquire/release/relaxed). Go gives you fewer footguns. - vs Rust
std::sync::atomic: Rust exposes the full memory ordering palette like C++. Go is intentionally simpler.
sync.Once, sync.Pool, the runtime scheduler itself, and most well-written concurrent caches.- For protecting more than one variable — use a Mutex. Atomic ops on two separate variables don't compose.
- For complex state transitions — CAS retry loops are tricky to get right. Mutex is clearer.
- For low-contention workloads — Mutex is fine and clearer to read.
- For 64-bit atomics on 32-bit ARM — alignment requirements bite. The typed API hides this; the function API doesn't.
- Mixing atomic and non-atomic access on the same variable is a data race even though one side is "atomic". The race detector catches this.
- The old function API requires alignment: a
uint64field in a struct must be 8-byte aligned. The typed API handles this automatically. - CAS retry loops can spin forever under high contention — back off or fall to a mutex.
atomic.Valuestores any value but the type cannot change after the firstStore.
The sync/atomic package provides lock-free operations for simple counters and flags. Faster than mutex for single-value operations.
import "sync/atomic"
type AtomicCounter struct {
count int64
}
func (ac *AtomicCounter) increment() {
atomic.AddInt64(&ac.count, 1) // thread-safe increment, no mutex needed
}
func (ac *AtomicCounter) getValue() int64 {
return atomic.LoadInt64(&ac.count) // thread-safe read
}
func main() {
var wg sync.WaitGroup
counter := &AtomicCounter{}
for range 10 {
wg.Add(1)
go func() {
defer wg.Done()
for range 1000 { counter.increment() }
}()
}
wg.Wait()
fmt.Println("Final:", counter.getValue()) // always 10000
}
// Without atomic/mutex: result would be random (race condition!)
// Atomic ops: AddInt64, LoadInt64, StoreInt64, CompareAndSwapInt64
- Request counters — track requests per second with
atomic.AddInt64— orders of magnitude faster than mutex - Boolean flags —
atomic.StoreInt32(&running, 0)as a shutdown signal checked from multiple goroutines - Lock-free counters — Prometheus client libraries store metric values atomically
- Compare-and-swap (CAS) — building lock-free data structures, optimistic concurrency, version checks
- atomic.Value — store/load any type atomically; perfect for hot-reloading config without locks
- Go 1.19+ atomic.Int64 / atomic.Bool — type-safe wrappers; preferred over the function-based API
35 Race Conditions & Deadlocks
A race condition happens when two or more goroutines access the same memory location concurrently, at least one of them writes, and there's no synchronization ordering the accesses. The result depends on scheduling and can change run to run. Symptoms: occasional wrong answers, corrupted data structures, "impossible" panics.
A deadlock is when goroutines are all waiting on each other (or on channels/locks no one will release) and nothing can make progress. The whole program — or a subset of it — freezes.
Go ships world-class tooling for both: the race detector (
go run -race, go test -race) and a runtime deadlock detector for the trivial case.- Counter increment: two goroutines both read counter=5, both write 6 → final value is 6 instead of 7.
- Map write: two goroutines write to
m[k]simultaneously → runtime panic "concurrent map writes". - Slice append: append from multiple goroutines without sync → corrupted slice header, lost values.
- Stale read: one goroutine reads a flag, the other writes — without sync, the reader may see the stale value forever due to CPU caching and compiler reordering.
- Self-deadlock: sending to an unbuffered channel with no receiver in the same goroutine.
- Forgotten close: a worker ranges over a channel that's never closed → range never exits → goroutine blocked forever.
- Lock-order inversion: goroutine A locks
XthenY; goroutine B locksYthenX. Eventually they cross — both wait forever. - Wait-without-send: goroutine waits on a channel send that never happens because the sender errored.
- WaitGroup forgot Add:
wg.Wait()returns immediately because the count was 0. - Mutex held during channel op: goroutine A holds mutex and tries to send on a channel; goroutine B is supposed to receive but first wants the mutex.
- Usage:
go test -race ./...,go run -race main.go,go build -race. - How it works: instruments memory accesses at compile time and tracks happens-before relationships at runtime.
- Output: when a race is detected, you get the goroutine IDs, stack traces of both the read and the write, and which mutexes were (or weren't) held.
- Cost: ~5–10× CPU and ~5–10× memory. Use in tests/staging, not production.
- False negatives: the detector finds races that actually happened in this run — it can't prove absence. Run your test suite many times.
- Industry standard: always run CI with
-race.
- vs Java: Java has no built-in race detector; tools like ThreadSanitizer or third-party libraries are needed. Go's is part of the toolchain.
- vs C++ ThreadSanitizer: Go's race detector is built on the same TSan runtime — same engine, integrated into
go test. - vs Rust: Rust prevents most data races at compile time via the borrow checker. Go catches them at runtime via the race detector — easier to learn, less safe.
- Whole-program deadlock detection: Go's runtime detects when every goroutine is blocked and panics with
fatal error: all goroutines are asleep - deadlock!. Partial deadlocks (only some goroutines stuck) needSIGQUITor pprof to diagnose.
-race in CI catches them before they bite real users. Knowing the canonical deadlock shapes (lock ordering, forgotten close, wait-without-send) is what keeps experienced Go programmers from creating them in the first place.- Always run
-race: in CI, in dev, in long-running stress tests. - Acquire locks in a consistent order across the entire codebase to avoid inversion.
- Use channels to transfer ownership rather than locks to share data.
- Use higher-level primitives like
errgroupinstead of raw goroutines. - Always have a way out: every long-running goroutine needs a context, timeout, or done channel.
- Use leak-detection tools like
uber-go/goleakin tests.
Race Conditions
A race condition occurs when multiple goroutines access shared data and at least one modifies it, without synchronization. Go has a built-in race detector.
// DETECT races: go run -race filename.go
// BUG: Race condition — unpredictable result
var count int
for range 10 {
go func() {
for range 1000 { count++ } // UNSAFE! multiple goroutines write
}()
}
// count might be 8523, 9101, 10000 — never predictable
// FIX: Use mutex
var mu sync.Mutex
for range 10 {
go func() {
for range 1000 {
mu.Lock()
count++
mu.Unlock()
}
}()
}
// count will always be 10000
Deadlocks
A deadlock happens when goroutines wait for each other forever. Classic cause: acquiring multiple locks in different order.
// DEADLOCK: goroutine 1 locks A then waits for B
// goroutine 2 locks B then waits for A
var mu1, mu2 sync.Mutex
go func() {
mu1.Lock()
fmt.Println("Goroutine 1 locked mu1")
time.Sleep(time.Second)
mu2.Lock() // waits for mu2... but goroutine 2 has it
mu1.Unlock(); mu2.Unlock()
}()
go func() {
mu2.Lock()
fmt.Println("Goroutine 2 locked mu2")
time.Sleep(time.Second)
mu1.Lock() // waits for mu1... but goroutine 1 has it
mu2.Unlock(); mu1.Unlock()
}()
// DEADLOCK! Both goroutines wait forever
// FIX: Always acquire locks in the SAME order
// Both goroutines should lock mu1 first, then mu2
Go's runtime detects deadlocks when ALL goroutines are blocked: fatal error: all goroutines are asleep - deadlock!. But it can't detect partial deadlocks where only some goroutines are stuck. Always use go run -race during development.
go test -racein CI — every PR runs with the race detector; catches concurrent map writes, unprotected counters- Lock ordering rule — document a strict order (e.g., "always lock orders.mu before users.mu") to prevent ABBA deadlocks
- Channel direction restrictions —
chan<- Tin producers,<-chan Tin consumers — compiler catches misuse - Avoid holding locks across function calls — minimize the critical section to prevent unknown reentrancy
- Use copy-on-write for shared state — atomically swap the whole struct with
atomic.Valueto avoid lock contention - Goroutine leak detection —
uber-go/goleakin tests catches goroutines that didn't shut down
36 Context
context.Context is one of the most important interfaces in Go — it carries deadlines, cancellation signals, and request-scoped values across API boundaries and goroutines. You create a root context with context.Background() (the standard root) or context.TODO() (placeholder for "I don't know which context to use yet"), then derive children:
WithCancel(parent)— returns a new context plus acancel()function. Callingcancel()closes the context'sDonechannel, propagating to all descendants.WithTimeout(parent, d)— like WithCancel but auto-cancels after durationd.WithDeadline(parent, t)— auto-cancels at a specific wall-clock time.WithValue(parent, key, value)— attaches a request-scoped value (use sparingly!).WithoutCancel/WithDeadlineCause/WithCancelCause(Go 1.21+) — modern variants for advanced patterns.
ctx is the first parameter of every function that does I/O.
type Context interface {
Deadline() (deadline time.Time, ok bool)
Done() <-chan struct{}
Err() error
Value(key any) any
}
Done()returns a channel that's closed when the context is cancelled. This is what youselecton.Err()returnsnilif not done, orcontext.Canceled/context.DeadlineExceededafter.Deadline()returns the deadline (if any) for downstream timeout calculations.Value()retrieves request-scoped data — use sparingly.
context.Background(), and every WithX call creates a child. When a parent is cancelled, the cancellation propagates to all descendants automatically. This means:
- Cancelling a request context cancels every downstream operation (DB queries, HTTP calls, sub-goroutines).
- Setting a 5-second timeout on a request automatically gives every nested call at most 5 seconds.
- A panic in one branch can be cleanly contained by cancelling its context.
- Forgetting to call
cancel()leaks resources — alwaysdefer cancel().
- vs Java thread interrupts: Java uses
Thread.interrupt()which sets a flag the thread checks. Go's context is explicit and granular — you can have multiple cancellation scopes per goroutine. - vs .NET
CancellationToken: Very similar in spirit. Both pass a token through method calls; both let you observe cancellation via a callback or channel. - vs JS
AbortController: Same idea, more recent invention. JS lacks the tree-of-cancellation propagation that Go provides automatically. - vs Python
asyncio.CancelledError: Python's cancellation is exception-based and tied to the event loop. Go's is value-based and works the same in any concurrency model. - Just an interface: Go's context is not a special runtime feature. It's an ordinary interface with channels and mutexes inside — anyone can implement a custom Context.
net/http, database/sql, os.exec, gRPC, file I/O in modern Go) takes a context as its first argument. Adopting the convention everywhere makes graceful shutdown, per-request timeouts, and resource cleanup trivial. It's also how you propagate request IDs, user identity, and trace IDs through your call stack without globals.- Always pass
ctxas the first parameter — never store it in a struct field unless you're a long-running service. - Always
defer cancel()after creating a derived context, even if it has a timeout. - Don't use
context.WithValuefor ordinary parameters — only for cross-cutting concerns like request ID, user, tracing. - Use typed keys for
WithValue:type ctxKey string; const userKey ctxKey = "user"— prevents collisions. - In long-running goroutines, watch
ctx.Done()in a select alongside your work channel. - Don't pass
nilcontexts — usecontext.TODO()if you genuinely don't have one. - Tests: use
context.Background()withWithTimeoutfor deterministic timeouts.
context.Context is an interface in Go's standard library (context package) that carries deadlines, cancellation signals, and request-scoped values across API boundaries and between goroutines. Think of it as a "bag" that travels with every request through your entire call chain, telling downstream functions "how long you have" and "should you stop?"
The Context Interface
// context.Context is an interface with 4 methods
type Context interface {
Deadline() (deadline time.Time, ok bool) // when this context will be cancelled (if set)
Done() <-chan struct{} // closed when context is cancelled — listen on this
Err() error // nil if not done; DeadlineExceeded or Canceled after
Value(key any) any // returns value for key, or nil
}
Context Types — Which Does What
// 1. Background — the root context, never cancelled
// Use in: main(), init(), tests, top-level starting point
ctx := context.Background() // empty context, no deadline, no values
// 2. TODO — placeholder when you're not sure which context to use yet
ctx = context.TODO() // same as Background but signals "I need to figure this out"
// 3. WithCancel — manual cancellation
// Returns a copy of parent + a cancel function
ctx, cancel := context.WithCancel(context.Background())
defer cancel() // ALWAYS defer cancel to free resources
// When you call cancel(), ctx.Done() channel closes
// All child contexts derived from ctx are also cancelled
// 4. WithTimeout — auto-cancels after duration
ctx, cancel = context.WithTimeout(context.Background(), 5*time.Second)
defer cancel() // cancel fires automatically after 5s, but defer is still needed
// 5. WithDeadline — auto-cancels at specific time
deadline := time.Now().Add(10 * time.Second) // absolute time
ctx, cancel = context.WithDeadline(context.Background(), deadline)
defer cancel()
// WithTimeout is just shorthand for WithDeadline(parent, time.Now().Add(d))
// 6. WithValue — attach request-scoped data
type ctxKey string // custom type avoids key collisions between packages
ctx = context.WithValue(ctx, ctxKey("requestID"), "abc-123")
// Retrieve: ctx.Value(ctxKey("requestID")).(string) → "abc-123"
Context Tree — Parent-Child Relationship
// Contexts form a TREE — cancelling parent cancels ALL children
//
// Background (root)
// |
// WithCancel (request)
// / \
// WithTimeout WithValue
// (DB query) (logging)
//
// If "request" is cancelled → DB query and logging are BOTH cancelled
// But cancelling "DB query" does NOT cancel "request" or "logging"
parentCtx, parentCancel := context.WithCancel(context.Background())
childCtx, childCancel := context.WithTimeout(parentCtx, 3*time.Second)
defer childCancel()
parentCancel() // cancels parentCtx AND childCtx
fmt.Println(childCtx.Err()) // context canceled (inherited from parent)
How to Use — Listening for Cancellation
// Pattern 1: select on ctx.Done()
func doWork(ctx context.Context) error {
select {
case <-ctx.Done(): // context cancelled or timed out
return ctx.Err() // context.Canceled or context.DeadlineExceeded
case result := <-longOperation():
fmt.Println(result)
return nil
}
}
// Pattern 2: check in loops
func processItems(ctx context.Context, items []string) error {
for _, item := range items {
if ctx.Err() != nil { // check before expensive work
return ctx.Err() // stop early if cancelled
}
process(item)
}
return nil
}
Real-World Example — HTTP Server
// When a client disconnects, the request context is cancelled automatically
// This propagates to ALL downstream work (DB, APIs, etc.)
func handleOrder(w http.ResponseWriter, r *http.Request) {
ctx := r.Context() // request context — cancelled if client disconnects
// Step 1: Query database (passes ctx — DB will abort if client gone)
user, err := db.GetUser(ctx, userID)
if err != nil { return }
// Step 2: Call payment API with 3s timeout
payCtx, cancel := context.WithTimeout(ctx, 3*time.Second)
defer cancel()
result, err := paymentAPI.Charge(payCtx, user, amount) // aborts after 3s
if err != nil { return }
// Step 3: Send notification (separate goroutine, inherits request context)
go notifier.Send(ctx, user, result) // cancelled if client disconnects
}
// In the DB layer — context flows through the entire chain
func (db *DB) GetUser(ctx context.Context, id int) (*User, error) {
return db.conn.QueryRowContext(ctx, // database/sql respects context
"SELECT * FROM users WHERE id = $1", id,
).Scan(&user) // if ctx cancelled → query aborted → resources freed
}
Real-World Example — Graceful Shutdown
func main() {
srv := &http.Server{Addr: ":8080"}
// Start server in goroutine
go srv.ListenAndServe()
// Wait for interrupt signal (Ctrl+C)
quit := make(chan os.Signal, 1)
signal.Notify(quit, os.Interrupt)
<-quit // blocks until signal received
// Give active requests 10 seconds to finish
ctx, cancel := context.WithTimeout(context.Background(), 10*time.Second)
defer cancel()
srv.Shutdown(ctx) // waits for handlers OR 10s timeout, whichever first
}
Why Context Exists in Go
- Goroutine cleanup — Go has no way to "kill" a goroutine from outside. Context is the cooperative cancellation mechanism: goroutines check
ctx.Done()and exit voluntarily - Resource leaks — Without context, if a client disconnects mid-request, the server keeps running DB queries, API calls, and goroutines for a response nobody will read
- Timeout propagation — A single request might call 5 services. Context ensures the overall deadline flows to all of them — if the top-level 10s deadline hits, everything stops
- No thread-local storage — Unlike Java/Python, Go has no thread-local variables. Context is how you pass request-scoped data (request IDs, auth tokens) through the call chain
- Always pass
context.Contextas the first parameter, namedctx - Never store context in a struct — pass it explicitly through function calls
- Use
context.Background()in main/init/tests (the root) - Use
context.TODO()when unsure — it's a placeholder to fix later - Always call
cancel()viadefer cancel()— prevents goroutine/memory leaks - Use
WithValuesparingly — only for request-scoped data (IDs, auth), not for passing function parameters
Full Example — WithCancel + WithValue + Goroutines
This example ties together everything: Background → WithCancel for manual cancellation, WithValue for request-scoped data, a worker goroutine listening on ctx.Done(), and a logging helper that extracts values from context.
package main
import (
"context"
"fmt"
"log"
"time"
)
// doWork runs in a goroutine, keeps working until context is cancelled
func doWork(ctx context.Context) {
for {
select {
case <-ctx.Done(): // fires when cancel() is called
fmt.Println("Work cancelled:", ctx.Err()) // "context canceled"
return // exit goroutine cleanly
default:
fmt.Println("Working...") // keep doing work
}
time.Sleep(500 * time.Millisecond) // simulate work interval
}
}
// logWithContext extracts request-scoped data from context for structured logging
func logWithContext(ctx context.Context, message string) {
requestID := ctx.Value("requestID") // extract value — returns any (interface{})
log.Printf("RequestID: %v - %v", requestID, message)
}
func main() {
// Step 1: Start with root context
ctx := context.Background() // empty root — no deadline, no values
// Step 2: Create cancellable context (manual control)
ctx, cancel := context.WithCancel(ctx)
// Unlike WithTimeout (auto-cancels after duration),
// WithCancel requires YOU to call cancel() when ready
// Step 3: Simulate heavy task that cancels when done
go func() {
time.Sleep(2 * time.Second) // simulate 2s of heavy work
cancel() // signal all goroutines: "stop!"
}()
// Step 4: Attach request-scoped values (chain of WithValue calls)
// Each WithValue wraps the previous context — forms a chain
ctx = context.WithValue(ctx, "requestID", "hdsjf3234324234")
ctx = context.WithValue(ctx, "name", "John")
ctx = context.WithValue(ctx, "IP", "192.168.1.100")
ctx = context.WithValue(ctx, "OS", "Linux")
// Step 5: Launch worker — it will print "Working..." until cancel() is called
go doWork(ctx)
// Step 6: Wait 3s so we can see the worker run then stop
time.Sleep(3 * time.Second)
// Step 7: Read values from context
requestID := ctx.Value("requestID") // values persist even after cancellation
if requestID != nil {
fmt.Println("Request ID:", requestID) // "Request ID: hdsjf3234324234"
}
logWithContext(ctx, "This is a test log message")
// Output: RequestID: hdsjf3234324234 - This is a test log message
}
// TIMELINE:
// 0.0s → doWork starts: "Working..."
// 0.5s → "Working..."
// 1.0s → "Working..."
// 1.5s → "Working..."
// 2.0s → cancel() called → doWork sees ctx.Done() → "Work cancelled: context canceled"
// 3.0s → main prints Request ID and log message
- WithTimeout / WithDeadline — use when you have a fixed time limit: "this DB query must finish in 5 seconds", "this API call gets 3 seconds max"
- WithCancel — use when cancellation depends on a condition, not time: "cancel when the heavy task finishes", "cancel when the user disconnects", "cancel when we find the first result"
context.TODO vs context.Background
Functionally identical — both create an empty, never-cancelled context. The difference is intent:
// Background — "I deliberately want the root context here"
// Use in: main(), init(), tests, or when you're the entry point
ctx := context.Background()
// TODO — "I don't know which context to use yet, I'll fix this later"
// Use when: you're refactoring, prototyping, or the API doesn't pass ctx yet
ctx = context.TODO()
// Both work exactly the same way with WithValue, WithCancel, etc.
ctx1 := context.WithValue(context.TODO(), "name", "John")
fmt.Println(ctx1.Value("name")) // "John"
ctx2 := context.WithValue(context.Background(), "city", "New York")
fmt.Println(ctx2.Value("city")) // "New York"
// So when should you use TODO?
// When you KNOW a proper context should be passed from a caller,
// but it's not available yet. TODO is a searchable marker:
// grep -r "context.TODO" . ← find all the places you need to fix
checkEvenOdd — Context Cancellation with Timeout
Shows how the same function behaves differently depending on whether the context is still alive or has timed out:
func checkEvenOdd(ctx context.Context, num int) string {
select {
case <-ctx.Done(): // context already cancelled?
return "Operation canceled" // don't bother doing work
default: // context still alive, proceed
if num%2 == 0 {
return fmt.Sprintf("%d is even", num)
}
return fmt.Sprintf("%d is odd", num)
}
}
func main() {
// With TODO — works fine, never cancelled
ctx := context.TODO()
fmt.Println(checkEvenOdd(ctx, 5)) // "5 is odd"
// With timeout — works while alive
ctx, cancel := context.WithTimeout(context.Background(), 1*time.Second)
defer cancel()
fmt.Println(checkEvenOdd(ctx, 10)) // "10 is even" (within 1s)
time.Sleep(3 * time.Second) // wait past the 1s timeout
fmt.Println(checkEvenOdd(ctx, 15)) // "Operation canceled" (timeout expired!)
// ctx.Done() channel is already closed, so select picks that case
}
// MISTAKE 1: Forgetting to cancel — leaks goroutines!
ctx, cancel := context.WithTimeout(ctx, 5*time.Second)
// missing: defer cancel() ← timer goroutine keeps running forever
// MISTAKE 2: Using context.Background() instead of passing parent
// This breaks the cancellation chain!
func handler(ctx context.Context) {
// WRONG: newCtx := context.WithTimeout(context.Background(), 5*time.Second)
// RIGHT: newCtx := context.WithTimeout(ctx, 5*time.Second) ← inherits parent
}
// MISTAKE 3: Using string keys for WithValue
// WRONG: ctx = context.WithValue(ctx, "userID", 123) ← any package can collide
// RIGHT: use unexported custom type as key
type contextKey string // unexported — only your package can create these keys
const userIDKey contextKey = "userID"
- HTTP request lifecycle —
r.Context()auto-cancels when client disconnects; pass it to all DB calls and downstream HTTP requests - gRPC calls — every gRPC method takes a context as first arg; deadline propagates across services
- Database queries —
db.QueryContext(ctx, ...)aborts query if context cancels; prevents zombie queries on disconnected clients - Trace/request IDs — store request ID in context; structured loggers extract it for every log line
- Graceful shutdown —
srv.Shutdown(ctx)with timeout context lets in-flight requests complete - Auth user / tenant — middleware injects authenticated user into context, handlers extract via typed key
37 GOMAXPROCS & Parallelism
GOMAXPROCS is a runtime knob that sets how many OS threads can execute Go code simultaneously. Default value (since Go 1.5) = number of logical CPUs visible to the process. Tunable at runtime via runtime.GOMAXPROCS(n) or before startup via the GOMAXPROCS environment variable. This setting is the dividing line between Go's two related but distinct concurrency concepts:
- Concurrency = structuring your program as multiple independent activities (goroutines).
- Parallelism = those activities actually executing simultaneously on multiple CPU cores.
GOMAXPROCS=1, you have concurrency without parallelism — many goroutines but only one runs at a time. With GOMAXPROCS=8 on an 8-core machine, you get full parallelism.
- G — a goroutine (lightweight, ~2 KB initial stack).
- M — an OS thread (machine).
- P — a processor (logical execution context). The number of Ps =
GOMAXPROCS.
- vs Node.js: Node has one event-loop thread (no parallelism for JS code). Go scales across all cores by default.
- vs Java threads: Java threads map 1:1 to OS threads, scheduled by the kernel. Go's M:N scheduler is far cheaper (millions of goroutines possible) and the runtime makes smart decisions about where to run them.
- vs Erlang's BEAM: Similar M:N approach with preemptive scheduling. Erlang has stronger preemption guarantees; Go's preemption is mostly cooperative with safe-point preemption (Go 1.14+).
- The cgroup awareness gap: Go's default does not respect Linux cgroup CPU limits. In a Kubernetes pod with a 0.5 CPU quota on a 64-core node, Go still sees 64 and starts 64 P's. Result: massive over-scheduling, lock contention, and latency spikes.
GOMAXPROCS to match your CPU quota, not the host's CPU count. The easiest fix is to import go.uber.org/automaxprocs, which reads the cgroup limits at startup and sets GOMAXPROCS accordingly. Mismatched values are one of the most common causes of unexplained latency spikes in containerized Go services. Another use case: if you're profiling a CPU-bound algorithm and want to compare scaling, manually set GOMAXPROCS=1, 2, 4, etc.- Current setting:
runtime.GOMAXPROCS(0)(zero means "just return current value"). - Number of CPUs:
runtime.NumCPU()— what the OS reports. - Goroutine count:
runtime.NumGoroutine()— useful for leak detection. - Scheduler trace:
GODEBUG=schedtrace=1000 ./myapp— emits scheduler stats every second. - Block/contention profiling:
runtime.SetBlockProfileRate,runtime.SetMutexProfileFraction+ pprof.
Concurrency is about dealing with multiple things at once (structure). Parallelism is about doing multiple things at once (execution). Go's scheduler multiplexes goroutines onto OS threads controlled by GOMAXPROCS.
import "runtime"
// GOMAXPROCS sets how many OS threads can execute Go code simultaneously
// Default: number of CPU cores
fmt.Println("CPUs:", runtime.NumCPU()) // e.g., 12
// Set to 1 = concurrency (goroutines take turns on 1 thread)
runtime.GOMAXPROCS(1)
// Set to N = parallelism (goroutines run on N threads simultaneously)
runtime.GOMAXPROCS(12) // use all 12 cores
// CPU-bound tasks benefit from more threads
func heavyTask(id int, wg *sync.WaitGroup) {
defer wg.Done()
fmt.Printf("Task %d starting\n", id)
for range 100_000_000 {} // CPU-heavy work
fmt.Printf("Task %d finished\n", id)
}
numThreads := runtime.NumCPU()
runtime.GOMAXPROCS(numThreads)
var wg sync.WaitGroup
for i := range numThreads {
wg.Add(1)
go heavyTask(i, &wg) // runs on all cores in parallel
}
wg.Wait()
Concurrency: 1 cook switching between 5 dishes. Parallelism: 5 cooks each making 1 dish. Go makes concurrency easy with goroutines. GOMAXPROCS controls how many goroutines can truly run in parallel. For I/O-bound work, concurrency alone is enough. For CPU-bound work, parallelism gives real speedup.
- Containers — Go pre-1.25 reads host CPU count, not cgroup limit; use
uber-go/automaxprocsto fix this in Kubernetes - CPU-bound tasks — image processing, compression, ML inference benefit from
GOMAXPROCS = NumCPU - I/O-bound tasks — database queries, HTTP calls don't need parallelism; concurrency alone is enough
- Reduce in noisy neighbors — set
GOMAXPROCS=2if you share a host with other CPU-hungry processes - Benchmark to find sweet spot — sometimes more cores hurt due to scheduling overhead; profile under realistic load
38 Stateful Goroutines
for { select { ... } } loop that processes one message at a time. No locks are needed because there's literally one goroutine touching the data, so concurrent access is impossible by construction.- An owning goroutine: launched once (often at startup), runs forever until shutdown.
- A request channel: typed messages representing operations (commands) on the state.
- Reply channels: embedded in each request, allowing the owner to send back a result.
- The select loop: handles each operation atomically; the state is never touched between iterations.
- Optional shutdown: close the request channel or watch a separate
donechannel.
type GetReq struct { Key string; Reply chan string }
type SetReq struct { Key, Val string }
func server(get chan GetReq, set chan SetReq) {
state := map[string]string{}
for {
select {
case r := <-get: r.Reply <- state[r.Key]
case r := <-set: state[r.Key] = r.Val
}
}
}
- vs Mutex-protected struct: Both serialize access. The stateful-goroutine version is serialized by construction — there's literally one reader/writer — so deadlocks from forgotten locks or order-inversion are impossible. The cost: every operation has channel-send overhead (~50 ns).
- vs Actor model (Akka, Erlang): Same idea, but built from raw goroutines and channels rather than a framework. No supervision trees, no mailbox management, no PID — just standard Go.
- vs Distributed actors: Stateful goroutines live in one process; for cross-machine actors, you'd add a network layer or use a real distributed framework.
- vs Channel of
func(): A simpler variant —chan func()and the owning goroutine just runs each function. Less type safety but very flexible.
- State needs to enforce multi-field invariants (a connection state machine, an in-memory queue, a session store, a rate limiter).
- You want trivial shutdown — close the input channel, the goroutine exits, the state is gone.
- You want request-level operations (like a tiny in-process service) without deploying anything.
- You want to minimize lock-related bugs in critical code paths.
http.Server's connection state machine, the Go runtime's scheduler heart, in-memory caches with TTL, in-process pub/sub.
- Per-op overhead: ~50–200 ns for the channel send + receive. Negligible for non-trivial operations, measurable in tight loops.
- One-at-a-time bottleneck: if operations are CPU-heavy, the owner becomes a single thread bottleneck — you'd need sharding (one goroutine per shard).
- Backpressure: the request channel can fill up; senders block. Choose the buffer size and timeout strategy carefully.
- Less obvious in stack traces: a panic in the owner crashes its goroutine; senders see channel closed. Add
recoverin the owner if needed.
Instead of sharing state with mutexes, let a single goroutine own the state and communicate via channels. This is the "share memory by communicating" philosophy.
type StatefulWorker struct {
count int
ch chan int
}
func (w *StatefulWorker) Start() {
go func() {
for {
select {
case value := <-w.ch:
w.count += value // only THIS goroutine modifies count
fmt.Println("Current count:", w.count)
}
}
}()
}
func (w *StatefulWorker) Send(value int) {
w.ch <- value // other goroutines communicate via channel
}
worker := &StatefulWorker{ch: make(chan int)}
worker.Start()
for i := range 5 {
worker.Send(i) // safe: no shared memory, just channels
time.Sleep(500 * time.Millisecond)
}
// Output: Current count: 0, 1, 3, 6, 10
- In-memory key-value cache — single goroutine owns the map; reads/writes go through a request channel
- Counter / metrics aggregator — multiple producers send increments, one goroutine accumulates
- Connection pool dispatcher — owner goroutine hands out connections from a pool via channels
- Game server state — one goroutine per room; players send actions, room broadcasts state updates
- Avoids mutex bugs — no shared state means no race conditions, no deadlocks; trade-off is channel overhead
- "Don't communicate by sharing memory; share memory by communicating" — Go's design philosophy
39 OS Signals
os/signal package — lets your Go program subscribe to Unix-style signals (SIGINT, SIGTERM, SIGHUP, SIGQUIT, etc.) and react to them. The traditional API: register a channel with signal.Notify(ch, syscall.SIGINT, syscall.SIGTERM) — when one of those signals arrives, the runtime sends the signal value on your channel. The modern API (Go 1.16+): signal.NotifyContext(parent, signals...) returns a context that's automatically cancelled when one of the signals arrives — perfect for graceful shutdown.- SIGINT (2): sent by Ctrl+C in the terminal. Default action: terminate.
- SIGTERM (15): "polite" termination request. Sent by Kubernetes when a pod is being shut down (with a default 30-second grace period before SIGKILL).
- SIGKILL (9): instant termination. Cannot be caught — the kernel kills the process immediately. No cleanup possible.
- SIGHUP (1): historically "terminal hang up", commonly used to request a config reload.
- SIGQUIT (3): Ctrl+\, dumps core. Go programs print a stack trace of all goroutines — incredibly useful for debugging.
- SIGUSR1 / SIGUSR2: user-defined; common for triggering custom actions like log rotation.
ctx, stop := signal.NotifyContext(context.Background(),
syscall.SIGINT, syscall.SIGTERM)
defer stop()
go server.Run(ctx) // pass ctx down everywhere
<-ctx.Done() // wait for signal
log.Println("shutting down...")
shutdownCtx, cancel := context.WithTimeout(
context.Background(), 30*time.Second)
defer cancel()
server.Shutdown(shutdownCtx)
This single pattern handles SIGINT and SIGTERM uniformly, propagates cancellation through the entire request graph, and gives you a bounded shutdown deadline.
- vs C signal handlers: In C, signal handlers run in a restricted async-signal context — you can only call a small list of "async-signal-safe" functions. Almost everything is forbidden, including
printf! Go converts signals into channel sends, so signal handling is normal Go code running in a normal goroutine. No reentrancy gotchas, nosigactionfiddling, no self-pipe tricks. - vs Java
Runtime.addShutdownHook: Java hooks run on JVM exit, but they can't be cancelled or chained the way Go contexts can. - vs Node.js
process.on('SIGINT'): Similar tosignal.Notifybut Node has only one event loop, so signal handlers fight with other event-loop work. - Windows: Windows doesn't have Unix signals natively. Go translates Ctrl+C and similar events into
os.Interruptfor portability.
- Buffer your signal channel:
make(chan os.Signal, 1)— if the channel is full when a signal arrives, the signal is dropped. - SIGKILL cannot be caught. No graceful path. Always design for the case where you might be killed.
- Bound your shutdown: always wrap the shutdown in
context.WithTimeoutso you exit before Kubernetes SIGKILLs you. - Don't
os.Exitin shutdown — it skips deferred cleanup. Letmainreturn naturally.
Handle OS signals (SIGINT, SIGTERM) for graceful shutdown — close database connections, flush logs, finish in-progress requests before exiting.
import (
"os"
"os/signal"
"syscall"
)
func main() {
sigs := make(chan os.Signal, 1)
done := make(chan bool, 1)
// Listen for interrupt (Ctrl+C) and terminate signals
signal.Notify(sigs, syscall.SIGINT, syscall.SIGTERM)
go func() {
sig := <-sigs // block until signal received
fmt.Println("\nReceived signal:", sig)
// Cleanup: close DB, flush logs, etc.
done <- true
}()
fmt.Println("Server running... Press Ctrl+C to stop")
<-done // wait for shutdown signal
fmt.Println("Graceful shutdown complete")
}
// Common signals:
// SIGINT (2) — Ctrl+C interrupt
// SIGTERM (15) — graceful termination request
// SIGHUP (1) — terminal closed / config reload
// SIGUSR1 (10) — user-defined signal
- Kubernetes graceful shutdown — k8s sends SIGTERM, waits
terminationGracePeriodSeconds(default 30s), then SIGKILL — your app must handle SIGTERM - SIGHUP for config reload — nginx-style: reload config without restarting (re-read file, swap atomically)
- Drain mode — on SIGTERM, stop accepting new requests, finish in-flight, then exit with 0
- signal.NotifyContext (Go 1.16+) — combines signal handling with context:
ctx, stop := signal.NotifyContext(ctx, os.Interrupt) - Buffered signal channel — always size ≥1; an unbuffered channel can drop signals if the goroutine isn't ready
- SIGPIPE — when writing to a closed connection; Go ignores this by default for networks
40 Rate Limiting
- Fixed window: simple counter that resets every N seconds. Easy to implement, but allows 2× burst at window boundaries.
- Sliding window: tracks timestamps within a rolling window. Smoother, but more memory.
- Token bucket: a bucket holding up to N tokens, refilled at rate R per second. Each operation consumes a token; if empty, wait or reject. Allows controlled bursts.
- Leaky bucket: a bucket with constant outflow rate. Smooths bursts into a steady stream — used for traffic shaping.
golang.org/x/time/rate package provides a production-quality token-bucket limiter that's the de-facto standard.
- Create:
limiter := rate.NewLimiter(rate.Limit(10), 50)— 10 events/sec, burst of 50. - Block until allowed:
err := limiter.Wait(ctx)— context-aware; respects cancellation. - Try without blocking:
if limiter.Allow() { ... }— returns immediately, true/false. - Reserve a slot:
r := limiter.Reserve(); ifr.OK(), thentime.Sleep(r.Delay())before proceeding. - Multiple tokens:
limiter.WaitN(ctx, n)— for operations that consume multiple tokens (e.g. bulk requests). - Update rate dynamically:
limiter.SetLimit(newRate)at runtime.
- vs naive
time.Sleepthrottling: sleep doesn't compose across goroutines, doesn't share fairness, and doesn't respect cancellation. Token bucket gives you a real budget that all goroutines can share. - vs Java/Spring rate limiters: Java has many libraries (Guava's
RateLimiter, Resilience4j) — same underlying algorithms, more verbose APIs. - vs Redis-based distributed limiters:
x/time/rateis in-process only. For limiting across multiple service instances, you need Redis or a centralized service. - vs API gateway rate limiting: Gateways (Kong, Envoy, nginx) can rate-limit inbound traffic;
x/time/rateis for limiting outbound calls or per-endpoint behavior in your code. - Context integration: Go's
limiter.Wait(ctx)blocks but cancels cleanly when the context is done — most other ecosystems don't have this elegance.
- Abusive clients that hammer your API.
- Runaway loops in your own code that shouldn't be calling a downstream 1000 times/second.
- Downstream overload — protect a slow database from a fast-growing user base.
- Third-party quota limits (Stripe, Twilio, OpenAI, GitHub) — turn "429 Too Many Requests" failures into clean in-process backpressure.
- Cost control — pay-per-request APIs can blow up your bill if you don't limit calls.
- Per-IP or per-user limit: a
map[string]*rate.Limiter(with TTL eviction) gives each client their own bucket. - Outbound API throttle: wrap an HTTP client with a limiter to stay under a vendor's rate limit.
- Worker pool throttle: limit how fast workers consume from a queue.
- Login attempt limit: slow down brute-force attacks.
- Log/metric sampling: drop excessive events instead of overwhelming your logging pipeline.
Rate limiting controls how frequently operations are allowed. Essential for APIs, preventing abuse, and resource management. Three classic algorithms:
1. Fixed Window Algorithm
Counts requests in a fixed time window. Simple but can allow burst at window boundaries.
type FixedWindowLimiter struct {
mu sync.Mutex
count int
limit int // max requests per window
window time.Duration // window size
resetTime time.Time // when current window ends
}
func (rl *FixedWindowLimiter) Allow() bool {
rl.mu.Lock()
defer rl.mu.Unlock()
now := time.Now()
if now.After(rl.resetTime) { // window expired, reset
rl.resetTime = now.Add(rl.window)
rl.count = 0
}
if rl.count < rl.limit {
rl.count++
return true // allowed
}
return false // denied
}
limiter := &FixedWindowLimiter{limit: 3, window: time.Second}
2. Token Bucket Algorithm
Tokens are added at a steady rate. Each request consumes a token. Allows short bursts up to bucket capacity.
type TokenBucket struct {
tokens chan struct{} // buffered channel = bucket
refillTime time.Duration
}
func NewTokenBucket(capacity int, refill time.Duration) *TokenBucket {
tb := &TokenBucket{
tokens: make(chan struct{}, capacity),
refillTime: refill,
}
for range capacity { tb.tokens <- struct{}{} } // fill bucket
go tb.startRefill()
return tb
}
func (tb *TokenBucket) startRefill() {
ticker := time.NewTicker(tb.refillTime)
defer ticker.Stop()
for range ticker.C {
select {
case tb.tokens <- struct{}{}: // add token if not full
default: // bucket full, skip
}
}
}
func (tb *TokenBucket) Allow() bool {
select {
case <-tb.tokens: return true // consumed a token
default: return false // no tokens available
}
}
rl := NewTokenBucket(5, time.Second) // 5 tokens, refill 1/sec
3. Leaky Bucket Algorithm
Processes requests at a fixed rate. Excess requests are queued (or dropped). Smoothest output rate of the three.
type LeakyBucket struct {
capacity int
leakRate time.Duration // time between leaks
tokens int
lastLeak time.Time
mu sync.Mutex
}
func (lb *LeakyBucket) Allow() bool {
lb.mu.Lock()
defer lb.mu.Unlock()
elapsed := time.Since(lb.lastLeak)
tokensToAdd := int(elapsed / lb.leakRate)
lb.tokens += tokensToAdd
if lb.tokens > lb.capacity { lb.tokens = lb.capacity }
lb.lastLeak = lb.lastLeak.Add(time.Duration(tokensToAdd) * lb.leakRate)
if lb.tokens > 0 { lb.tokens--; return true }
return false
}
- API gateways — protect downstream services from traffic spikes, enforce per-user/per-API-key quotas
- Use
golang.org/x/time/rate— battle-tested token bucket implementation; don't reinvent it - Login throttling — limit failed login attempts per IP to slow brute-force attacks
- Outbound rate limit — when calling rate-limited third-party APIs (Stripe, Twilio, OpenAI), throttle to stay under their quota
- Distributed rate limiting — Redis-backed token bucket for multi-instance services (single-instance limits don't work behind a load balancer)
- Burst tolerance — token bucket allows occasional bursts; leaky bucket smooths everything to a steady rate
41 Generics (Go 1.18+)
- Generic function:
func Min[T constraints.Ordered](a, b T) T { if a < b { return a }; return b } - Generic type:
type Stack[T any] struct { items []T } - Generic method: the receiver carries the type parameter —
func (s *Stack[T]) Push(v T).
T constraints.Ordered) is itself an interface that lists the types or methods T must support.
any— alias forinterface{}; allows literally any type.comparable— built-in; allows types that support==and!=.- Type union:
type Number interface { int | int64 | float32 | float64 }— only these exact types. - Approximation (
~T):type Number interface { ~int | ~float64 }— also matches custom types whose underlying type isintorfloat64. - Method requirements: a constraint can list method signatures the type must have, just like a normal interface.
golang.org/x/exp/constraintsprovidesOrdered,Integer,Float,Signed,Unsigned,Complex.
- vs Java/C# generics: Java uses type erasure — at runtime there's no
List<String>, justList. C# uses runtime reified generics (everyList<T>is a distinct runtime type). Go uses a hybrid called "GC-shape monomorphization": types with the same memory layout share generated code, but each shape is checked at compile time. - vs C++ templates: C++ generates a fresh function for every type combination — fast at runtime, slow to compile, opaque error messages. Go's approach compiles fast and produces clear errors.
- vs Rust generics + traits: Rust does full monomorphization; constraints are nominal traits. Go's approach is closer in spirit but simpler.
- vs pre-generics Go: Before 1.18, the only options were
interface{}+ type assertions (loses type safety) or code generation viago generate. Generics replace both for the cases they fit.
Stack[T], Set[T], LinkedList[T]) and generic algorithms (Map, Filter, Reduce, Sort) without the runtime cost or type-assertion ceremony of interface{}. The Go 1.21 standard library added slices, maps, and cmp packages with generic helpers (slices.Sort, slices.Contains, maps.Keys, cmp.Compare). Use generics when the alternative is duplicating identical code per type — but not when a plain interface would do the job.- If an interface works: a function taking
io.Readerdoesn't need a type parameter. - For one-off code: a generic function used in only one place is just complexity for no benefit.
- For the "any type" use case:
anyby itself rarely justifies generics — if you don't constrain T, it's the same asinterface{}. - For methods with both pointer and value receivers on a generic type — Go has restrictions here.
- Performance: generics aren't always faster than interfaces; benchmark first.
- Sort:
slices.Sort([]int{3, 1, 2})instead ofsort.Ints+sort.Float64s+sort.Slice. - Functional helpers:
Map[T, U](s []T, f func(T) U) []U— finally clean. - Container libs: typed sets, ordered maps, LRU caches without
any. - Test helpers:
AssertEqual[T comparable](t *testing.T, got, want T).
// Generic function
func Min[T constraints.Ordered](a, b T) T { // T must support < operator
if a < b { return a } // works for any Ordered type
return b
}
Min(3, 5) // 3 (type inferred)
Min("a", "b") // "a"
// Type constraints
type Number interface {
~int | ~int64 | ~float32 | ~float64 // ~ means "underlying type"
}
// Generic struct
type Stack[T any] struct { items []T } // T is the element type
func (s *Stack[T]) Push(item T) { s.items = append(s.items, item) } // add to top
func (s *Stack[T]) Pop() (T, bool) {
var zero T // zero value of type T
if len(s.items) == 0 { return zero, false } // empty stack
item := s.items[len(s.items)-1] // get last element
s.items = s.items[:len(s.items)-1] // shrink slice
return item, true
}
// Generic Map/Filter
func Map[T, U any](s []T, fn func(T) U) []U { // transform T slice into U slice
r := make([]U, len(s)) // pre-allocate result
for i, v := range s { r[i] = fn(v) } // apply fn to each element
return r
}
slicespackage (Go 1.21+) —slices.Contains,slices.Sort,slices.Indexall use genericsmapspackage —maps.Keys,maps.Values,maps.Clonework for any map type- Generic data structures — type-safe stacks, queues, linked lists, sets without
interface{}casts - Database query helpers —
QueryOne[User](db, "SELECT...")returns a typed result - Avoid generics for one-off code — concrete types are clearer and compile faster
- Constraints package —
constraints.Ordered,Number,Signedfor numeric type constraints
42 Closures & Higher-Order Functions
- A closure is a function value that captures variables from its surrounding lexical scope and keeps them alive for as long as the closure exists. The captured variables are shared by reference — multiple closures referencing the same variable see each other's writes.
- A higher-order function is one that takes another function as an argument, returns a function as a result, or both.
- Capture: any variable referenced inside an anonymous function but defined outside is captured. Captured variables move from the stack to the heap if needed (escape analysis decides).
- By reference: closures capture the variable, not its current value. If the outer function modifies the variable later, the closure sees the new value.
- Lifetime extension: a captured variable lives as long as any closure references it — the GC keeps it alive.
- Shared state: multiple closures over the same variable share state (useful for counters, caches).
- Decorator/middleware:
func Logger(next http.Handler) http.Handler { return http.HandlerFunc(func(w, r) { log...; next.ServeHTTP(w, r) }) }. - Functional options:
NewServer(WithTimeout(5*time.Second), WithRetries(3)). EachWith*returns a closure that mutates the server. - Factory:
func multiplier(factor int) func(int) int { return func(n int) int { return n * factor } }. - Lazy init:
var get = sync.OnceValue(func() *DB { return connect() }). - Comparators:
slices.SortFunc(s, func(a, b Item) int { return a.ID - b.ID }). - Retry/backoff wrappers: a function that takes a function and retries it.
- Memoization: a closure that caches results in a captured map.
- vs Java: Java lambdas can only capture effectively final variables — you can't modify a captured variable. Go closures can read AND write captured variables freely.
- vs JavaScript: Same model — capture by reference, mutation visible to all closures. JS programmers feel at home.
- vs Python: Python has the same capture-by-reference behavior, with the same loop-variable footgun.
- vs C# delegates: Closures, but C# captures by reference for locals and by value for value-types — confusing. Go is uniform.
- The famous loop-variable bug (pre-Go 1.22):
for _, v := range slice { go func() { use(v) }() }— all goroutines saw the SAMEvbecause the loop reused one variable. Go 1.22 fixed this by giving each iteration its own copy.
- Memory leaks via long-lived captures: a closure stored globally keeps every captured variable alive. Don't capture giant slices accidentally.
- Loop variable capture in pre-1.22 code — wrap with a function call passing the value, or shadow inside the loop.
- Hidden allocations: closures that escape to the heap have a tiny per-call cost. Usually negligible, sometimes measurable in tight loops.
- Concurrent mutation: if multiple goroutines close over the same variable, you need synchronization.
// Closure — captures variables from outer scope
func counter() func() int { // returns a function
count := 0 // captured: lives as long as closure
return func() int { count++; return count } // closure over count
}
next := counter() // each call gets its own count
next() // 1
next() // 2
// Factory pattern
func multiplier(factor int) func(int) int { // returns configured func
return func(n int) int { return n * factor } // factor is captured
}
double := multiplier(2) // specialised multiplier
double(5) // 10
- HTTP middleware —
func authMiddleware(next http.Handler) http.Handlercloses over the wrapped handler - Functional options —
WithPort(8080)returns a closure that mutates a*Server - Sort comparators —
sort.Slice(users, func(i,j int) bool { return users[i].Age < users[j].Age }) - Retry/circuit-breaker wrappers — wrap a function call in retry logic, return a new function
- Lazy evaluation — defer expensive computation until needed:
compute := func() int { ... } - Avoid capture bugs — pre-1.22, loop variables in closures referenced the same address; pass as argument instead
43 Defer, Panic, Recover
defer— schedules a function call to run when the surrounding function returns (whether normally, via early return, or due to a panic). Multiple defers run in LIFO (stack) order.panic— aborts normal execution. The current function stops, deferred functions run, and the panic propagates up the stack until somethingrecovers it or the program crashes with a stack trace.recover— only valid inside a deferred function. Catches a panic, returns the panic value, and lets execution resume (in the deferring function).
- Arguments evaluated immediately, body runs at return:
defer fmt.Println(x)captures the current value ofx, even ifxchanges later. - LIFO order: if you defer
A(),B(),C(), they run in the order C → B → A. - Always runs: on normal return, early return, AND on panic — the only thing that skips defers is
os.Exit. - Cost: ~30 ns per defer in modern Go (used to be more); generally negligible.
- Common idioms:
defer file.Close(),defer mu.Unlock(),defer tx.Rollback(),defer wg.Done(),defer cancel().
- Panic causes: nil deref, index out of range, divide by zero, send on closed channel, type assertion failure, explicit
panic("..."). - Effect: the function unwinds, deferred functions run in LIFO order, and the panic continues up the call stack.
- Recover:
defer func() { if r := recover(); r != nil { /* handle */ } }()— must be inside a deferred function. - If unrecovered: the program prints a stack trace of all goroutines and exits with code 2.
- Goroutine isolation: a panic in one goroutine does NOT propagate to others — but it still kills the whole process unless recovered. Always recover at the top of every long-running goroutine.
- Returning values from recover: use named returns to set them inside the deferred recoverer.
defervs RAII (C++): RAII tracks resource ownership via destructors.deferis explicit and one-line — easier to read, easier to forget. RAII works on every scope;deferonly on function return.defervsfinally(Java/Python/C#): Go's defer is colocated with the resource acquisition (right next toos.Open) instead of stuck at the bottom of the function. Much cleaner for multi-resource code.defervsusing(C#) /with(Python): Similar block-scoped pattern, but defer works on function scope.panic/recovervs exceptions: Superficially similar but used very differently. Go programmers do not use panic for ordinary error flow — that's what theerrorreturn value is for. Panics are reserved for "the program is broken" situations: programmer errors, impossible states, unrecoverable corruption.
defer guarantees cleanup even on early returns, panics, and errors — closing files, releasing locks, rolling back transactions, decrementing wait groups, calling cancel functions. recover is mainly used in HTTP/RPC server middleware to keep one bad request from crashing the entire process: a recovery middleware wraps every handler with defer recover(), logs the panic, and returns 500 to the client. Used correctly, this trio makes Go code both robust and concise.- Programmer error: impossible-state assertions where continuing would corrupt data.
- Init-time failures: a critical config file missing at startup — let the program crash loudly.
- Unrecoverable corruption: internal data structure invariants violated.
- Library APIs: some APIs (like
regexp.MustCompile) panic by design when given invalid input known at compile time. - NOT for normal error flow: use
errorreturn values instead.
// defer — executes when function returns (LIFO order)
f, _ := os.Open("file.txt") // open file, ignoring error for brevity
defer f.Close() // guaranteed cleanup
// panic — unrecoverable error (use RARELY)
panic("something terrible happened") // unwinds stack, prints trace
// recover — catch panics (ONLY inside deferred func)
defer func() { // must be deferred to catch panics
if r := recover(); r != nil { // r is the panic value
fmt.Println("Recovered:", r)
}
}()
// Real-world: recovery middleware in HTTP servers
func recoveryMiddleware(next http.HandlerFunc) http.HandlerFunc { // wraps a handler
return func(w http.ResponseWriter, r *http.Request) {
defer func() { // set up panic recovery for this request
if err := recover(); err != nil { // caught a panic
http.Error(w, "Internal Error", 500) // return 500 to client
}
}()
next(w, r) // call the actual handler
}
}
- defer file/connection close —
defer f.Close(),defer conn.Close(),defer rows.Close()— guaranteed cleanup even on error - defer mutex unlock —
mu.Lock(); defer mu.Unlock()— release lock even if function panics - defer span finish — distributed tracing:
defer span.End()records duration regardless of return path - panic for unrecoverable bugs only — programmer errors (impossible state), not user errors (return errors instead)
- recover in HTTP middleware — every web server should have one to convert panics to 500 responses without crashing the process
- defer order is LIFO — last deferred runs first; useful for nested resource cleanup
- Careful with deferred args —
defer fmt.Println(time.Now())evaluatestime.Now()NOW, not at defer time
44 Linked List
- Singly linked list — each node has a
Nextpointer; traversal is one-way; head is the entry point. - Doubly linked list — each node has both
NextandPrev; bidirectional traversal; supports O(1) deletion if you have a node pointer. - Circular linked list — the last node points back to the head, useful for round-robin or buffer-like structures.
container/list (a doubly linked list), but most code defines its own ~30-line struct.
- Insert at head: create new node, point its Next to current head, update head pointer. O(1).
- Insert at tail: walk to end (O(n)) or use a tail pointer (O(1)).
- Delete a node: requires the previous node's pointer in singly linked (or doubly linked direct deletion).
- Reverse: iterate, flipping each Next pointer — classic interview question.
- Cycle detection: Floyd's tortoise and hare (slow + fast pointer).
- Find middle: two-pointer technique, slow advances 1, fast advances 2.
- Merge two sorted lists: walk both, take smaller, advance.
- vs Go slice: Slices are contiguous in memory — vastly better cache locality. For 99% of "list of items" use cases, a slice destroys a linked list in performance, even though theoretical Big-O suggests otherwise.
- vs Java
LinkedList: Same idea, similar API. Java'sArrayListusually wins, mirroring Go. - vs C linked list: Go has GC, so you can't accidentally leak nodes. C requires manual
free()on every removed node. - vs
container/list: Go's stdlib version usesanyfor values, requiring type assertions. With generics (Go 1.18+), a customtype List[T any]is type-safe.
- LRU caches: a doubly linked list + map gives O(1) get/put with eviction order tracking. Used in many production cache libraries.
- Free lists: a list of available slots in a pool — push/pop without copying.
- Intrusive data structures: embed list pointers directly in domain structs to participate in multiple lists at once.
- Frequent splice / move operations: if you're constantly moving items between lists, linked lists do it in O(1).
- Learning tool: implementing one yourself is the cleanest way to learn Go's pointer semantics, generics, and how the GC follows pointer graphs.
- "I just need a list of items" — use a slice.
- "I need fast random access" — use a slice.
- "I need to sort or search" — use a slice; sort is much faster on contiguous memory.
- "I'm storing primitives or small structs" — slices win by orders of magnitude due to cache effects.
type Node[T any] struct { // generic node for any type T
Val T // the data stored in this node
Next *Node[T] // pointer to next node (nil if last)
}
type LinkedList[T any] struct {
Head *Node[T] // first node (nil if empty)
Size int // track length
}
// Prepend — O(1)
func (ll *LinkedList[T]) Prepend(val T) {
ll.Head = &Node[T]{Val: val, Next: ll.Head} // new node points to old head
ll.Size++
}
// Append — O(n)
func (ll *LinkedList[T]) Append(val T) {
node := &Node[T]{Val: val} // new tail node
if ll.Head == nil { ll.Head = node; ll.Size++; return } // empty list
curr := ll.Head // start traversal from head
for curr.Next != nil { curr = curr.Next } // walk to last node
curr.Next = node; ll.Size++ // attach new node at end
}
// Reverse — O(n)
func (ll *LinkedList[T]) Reverse() {
var prev *Node[T] // becomes new tail (nil)
curr := ll.Head // current node being processed
for curr != nil {
next := curr.Next // save next before overwriting
curr.Next = prev // reverse the pointer
prev = curr // advance prev
curr = next // advance curr
}
ll.Head = prev // prev is now the new head
}
// Print: 0 -> 1 -> 2 -> nil
func (ll *LinkedList[T]) Print() {
for c := ll.Head; c != nil; c = c.Next { // walk the list
fmt.Printf("%v -> ", c.Val)
}
fmt.Println("nil") // mark end of list
}
container/list— Go's stdlib doubly-linked list; useful when you actually need O(1) middle insertion- LRU cache — doubly-linked list + map; move accessed nodes to front, evict from back (groupcache, hashicorp/golang-lru)
- Undo stack — text editors, transaction logs — append to head, traverse for history
- Most of the time, use a slice — slices are faster than linked lists for almost everything due to CPU cache locality
- Free lists — runtime allocator uses linked lists of free memory chunks
- Job queues — when you need stable iteration plus middle removal (rare; usually a channel suffices)
45 Stack & Queue
- Stack — LIFO (Last In, First Out): add and remove only at the top. Like a stack of plates: the last plate you put on is the first you take off. Operations: Push, Pop, Peek.
- Queue — FIFO (First In, First Out): add at the back, remove from the front. Like a line at a checkout. Operations: Enqueue, Dequeue, Peek.
- Deque (double-ended queue): both — operations at both ends. Used for things like sliding-window algorithms.
- Priority queue: dequeue order is by priority, not insertion. Implemented with a heap; Go's
container/heapprovides one.
[]T using append for push/enqueue.
type Stack[T any] struct { items []T }
func (s *Stack[T]) Push(v T) { s.items = append(s.items, v) }
func (s *Stack[T]) Pop() (T, bool) {
var zero T
if len(s.items) == 0 { return zero, false }
i := len(s.items) - 1
v := s.items[i]
s.items = s.items[:i]
return v, true
}
Push and Pop are both amortized O(1); the slice grows in geometric jumps as needed.
q = q[1:] is O(1) but holds the backing array; q = append(q[:0], q[1:]...) is O(n).
- Slice trick (simple): dequeue with
q = q[1:]— but the backing array grows forever; periodically copy to reset. - Circular buffer: fixed-size array with head/tail indexes that wrap around. O(1) on both ends, no allocations after initial sizing.
- Two-stack queue: use two stacks, in and out; when out is empty, pop everything from in to out. Amortized O(1).
- Channel as queue: for concurrent producer/consumer, a buffered channel is a thread-safe queue.
- vs Java
Deque/ArrayDeque/Stack: Java has dedicated classes; Go does not — slices already handle stacks perfectly. Java'sStackis famously deprecated in favor ofDeque. - vs Python
collections.deque: Python's deque is a high-performance double-ended queue. Go has no equivalent in stdlib; roll your own circular buffer or use a third-party. - vs
container/list: Linked-list based, O(1) on both ends, but slower per op due to allocations and cache misses; usesanyrequiring type assertion. - Channels as queues: for concurrency-aware queues, a buffered channel is the idiomatic Go choice — no manual locking.
container/list and casting any on every access.- Browser back/forward: two stacks.
- Function call stack: a stack of activation records.
- JSON/HTML parsers: stack of open tags/objects.
- BFS in a graph: queue of frontier nodes.
- Job queue worker: channel as a thread-safe FIFO.
- Sliding window maximum: deque holding indexes.
// Stack (LIFO)
type Stack[T any] struct { items []T } // backed by a slice
func (s *Stack[T]) Push(item T) { s.items = append(s.items, item) } // add to top
func (s *Stack[T]) IsEmpty() bool { return len(s.items) == 0 } // check before Pop
func (s *Stack[T]) Pop() (T, bool) {
var zero T // safe zero to return on failure
if s.IsEmpty() { return zero, false } // nothing to pop
idx := len(s.items) - 1 // index of top element
item := s.items[idx] // save it before shrinking
s.items = s.items[:idx] // remove top element
return item, true
}
// Queue (FIFO)
type Queue[T any] struct { items []T } // backed by a slice
func (q *Queue[T]) Enqueue(item T) { q.items = append(q.items, item) } // add to back
func (q *Queue[T]) Dequeue() (T, bool) {
var zero T // safe zero to return on failure
if len(q.items) == 0 { return zero, false } // empty queue
item := q.items[0] // get front element
q.items = q.items[1:] // remove front element
return item, true
}
// Classic: valid parentheses
func isValid(s string) bool {
stack := &Stack[rune]{} // stack of opening brackets
pairs := map[rune]rune{')': '(', ']': '[', '}': '{'} // closing → opening
for _, ch := range s { // iterate each character
if ch == '(' || ch == '[' || ch == '{' {
stack.Push(ch) // push opening bracket
} else {
top, ok := stack.Pop() // must match last opening
if !ok || top != pairs[ch] { return false } // mismatch
}
}
return stack.IsEmpty() // valid only if all brackets matched
}
- Stacks for parsers — JSON/HTML/YAML parsers track open brackets/tags on a stack
- Stacks for backtracking — DFS, undo operations, expression evaluators (postfix/infix)
- Queues for BFS — shortest-path algorithms, level-order tree traversal
- Channels as queues — Go channels are essentially thread-safe queues; use them when you need synchronization
- Slice-backed queue is wasteful —
q[1:]doesn't free memory; use a ring buffer for high-throughput queues - Browser history / call stack — your program's runtime call stack is literally a stack data structure
46 Binary Tree / BST
- Binary tree — a hierarchical structure where each node has up to two children, conventionally called left and right. No ordering rules; just shape.
- Binary search tree (BST) — a binary tree with an extra invariant: every node's left subtree contains smaller keys, right subtree contains larger (or equal) keys. This ordering makes search, insert, and delete O(log n) on average.
- Pre-order (root → left → right): used for serializing a tree, copying, prefix expressions.
- In-order (left → root → right): for a BST, this visits keys in sorted order!
- Post-order (left → right → root): used for deleting/freeing trees, postfix expressions, computing subtree sums.
- Level-order (BFS): visit nodes by depth using a queue.
- Insert: walk down comparing the key, attach new node at the first
nilchild. O(log n) average, O(n) worst. - Search: walk down comparing keys until found or
nil. Same complexity. - Delete: three cases — leaf (just remove), one child (replace with child), two children (replace with in-order successor or predecessor).
- Min/Max: walk all the way left (min) or right (max).
- Range query: in-order traversal with bounds, very efficient on balanced trees.
- Successor/Predecessor: find the next/prev key in sorted order without scanning.
- Plain BST: degenerates to O(n) on sorted input (becomes a linked list).
- AVL tree: self-balancing via height tracking; strict O(log n).
- Red-black tree: self-balancing via color invariants; used in Java's
TreeMap, C++'sstd::map, Linux scheduler. - B-tree / B+-tree: wide branching factor; the basis of every database index.
- Trie: prefix tree for strings; used in autocomplete, IP routing.
- Heap: a tree where each parent ≤ children (min-heap) or ≥ (max-heap); the basis of priority queues.
- Segment tree / Fenwick tree: for range sum / range min queries with updates.
- vs hash map: Maps are unordered, O(1) average. BSTs are ordered, O(log n). BSTs win when you need range queries, sorted iteration, or "next/previous" lookups.
- vs Java
TreeMap: Java has a built-in red-black tree map; Go does not — you'd implement one yourself or use a library. - vs C++
std::map: Same idea — sorted, O(log n) — but Go has nothing equivalent in stdlib. - vs database B-tree indexes: Same family of structures; databases use shallower trees with high fanout to minimize disk seeks.
type TreeNode struct {
Val int // node value
Left, Right *TreeNode // child pointers (nil if absent)
}
// Insert — O(log n) avg
func insert(node *TreeNode, val int) *TreeNode {
if node == nil { return &TreeNode{Val: val} } // base: create leaf
if val < node.Val { node.Left = insert(node.Left, val) } // go left
else if val > node.Val { node.Right = insert(node.Right, val) } // go right
return node // return unchanged node (or duplicate)
}
// In-order (sorted output for BST)
func inOrder(n *TreeNode, res *[]int) {
if n == nil { return } // base case
inOrder(n.Left, res) // visit left subtree first
*res = append(*res, n.Val) // then visit current node
inOrder(n.Right, res) // then visit right subtree
}
// Level-order (BFS)
func levelOrder(root *TreeNode) [][]int {
if root == nil { return nil } // empty tree
var result [][]int // each inner slice is one level
queue := []*TreeNode{root} // BFS queue starts with root
for len(queue) > 0 {
sz := len(queue) // nodes at current level
var level []int // collect values for this level
for i := 0; i < sz; i++ {
n := queue[0]; queue = queue[1:] // dequeue front
level = append(level, n.Val) // record value
if n.Left != nil { queue = append(queue, n.Left) } // enqueue children
if n.Right != nil { queue = append(queue, n.Right) }
}
result = append(result, level) // store completed level
}
return result
}
// Max depth
func maxDepth(n *TreeNode) int {
if n == nil { return 0 } // nil node contributes 0 depth
l, r := maxDepth(n.Left), maxDepth(n.Right) // depth of each subtree
if l > r { return l + 1 } // +1 for current node
return r + 1
}
- Database indexes — B-trees (balanced multi-way BST variants) power index lookups in PostgreSQL, MySQL, MongoDB
- Filesystem inode trees — ext4, XFS use B-trees to look up files within directories quickly
- Auto-complete / search suggestions — tries (prefix trees) for fast prefix matching
- Expression / AST trees — compilers, query planners, template engines build trees of operators
- Decision trees — ML libraries (random forests, gradient boosting) use them for classification
- Use stdlib first — for ordered maps, reach for
github.com/google/btreeinstead of writing your own
47 Graph
- Directed — edges have a direction (Twitter "follows", web links, dependencies).
- Undirected — edges are two-way (Facebook friends, train tracks).
- Weighted — edges carry a number (distance, cost, capacity).
- DAG — Directed Acyclic Graph; no cycles. Build systems, dependency graphs.
- Bipartite — vertices split into two sets, edges only between sets.
- Adjacency list:
map[NodeID][]NodeIDormap[NodeID][]Edge. Best for sparse graphs (few edges). Memory: O(V+E). Iterate neighbors fast. - Adjacency matrix:
[N][N]boolor[N][N]int. Best for dense graphs or when you need O(1) edge lookup. Memory: O(V²). - Edge list: just a slice of
(from, to, weight). Used by algorithms like Kruskal's MST. - Compressed sparse row (CSR): packed array for very large graphs.
map[K][]K is almost a one-liner.
- BFS (breadth-first search): queue-based traversal. Finds shortest paths in unweighted graphs.
- DFS (depth-first search): stack-based or recursive. Detects cycles, computes connected components, topological sort.
- Dijkstra: shortest path with non-negative weights. Uses a priority queue.
- Bellman-Ford: shortest path with negative weights; detects negative cycles.
- A*: heuristic-guided shortest path; used in pathfinding (games, GPS).
- Topological sort: linear ordering of a DAG; used in build systems, course prerequisites.
- Union-Find / DSU: tracks connected components incrementally.
- Kruskal / Prim: minimum spanning tree.
- Tarjan / Kosaraju: strongly connected components.
- Max flow: Ford-Fulkerson, Edmonds-Karp.
- vs trees: A tree is a graph with no cycles and exactly one path between any two nodes. Every tree is a graph; not every graph is a tree.
- vs other languages: No built-in graph type in Go's stdlib. The community library
gonum.org/v1/gonum/graphcovers heavy-duty needs (typed graphs, parsers, layouts, algorithms). - vs Python NetworkX: NetworkX has 100+ algorithms out of the box; Go's gonum is smaller but well-designed.
- vs database graph engines (Neo4j): in-process graphs vs persistent graph databases. Use Neo4j when the graph doesn't fit in memory or needs ACID.
- Forgetting visited tracking in DFS/BFS — leads to infinite loops on cycles.
- Recursion depth in DFS on huge graphs — switch to an explicit stack.
- Map iteration order is randomized — output may differ between runs unless you sort.
- Edge weights: Dijkstra is incorrect for negative weights — use Bellman-Ford.
type Graph struct {
adj map[string][]string // adjacency list
directed bool // true = directed, false = undirected
}
func (g *Graph) AddEdge(from, to string) {
g.adj[from] = append(g.adj[from], to) // add forward edge
if !g.directed { g.adj[to] = append(g.adj[to], from) } // add reverse for undirected
}
// BFS
func (g *Graph) BFS(start string) []string {
visited := map[string]bool{start: true} // mark start as visited
queue := []string{start} // BFS frontier
var result []string // visited order
for len(queue) > 0 {
node := queue[0]; queue = queue[1:] // dequeue front
result = append(result, node) // record visit
for _, nb := range g.adj[node] {
if !visited[nb] { visited[nb] = true; queue = append(queue, nb) } // enqueue unvisited
}
}
return result
}
// DFS (recursive)
func (g *Graph) DFS(start string) []string {
visited := make(map[string]bool) // track visited nodes
var result []string // accumulate visit order
var dfs func(string) // declare before defining (recursive closure)
dfs = func(n string) {
visited[n] = true // mark before recursing
result = append(result, n) // record visit
for _, nb := range g.adj[n] {
if !visited[nb] { dfs(nb) } // recurse into unvisited
}
}
dfs(start) // kick off from start node
return result
}
- Service dependency graphs — DAGs for microservices, deployment pipelines (Argo, Airflow, Tekton)
- Build systems — Bazel, Make, Go's own build graph use topological sort to order compile steps
- Social networks — friends-of-friends queries, recommendation systems use BFS/DFS
- Routing — Dijkstra's algorithm for shortest path (Maps, network packet routing)
- Cycle detection — detecting circular dependencies in import graphs, tasks queues, ORM relationships
- Library —
gonum.org/v1/gonum/graphfor serious graph work (algorithms, parsers, layouts)
48 Hash Table (from scratch)
map by hand. A hash table consists of:
- An array of buckets (slots).
- A hash function that turns a key into a bucket index — usually
hash(key) % bucketCount. - A collision-handling strategy for when two keys hash to the same bucket.
- A resize policy for when the table fills up beyond a load factor (typically 0.7–0.8).
- Separate chaining: each bucket holds a linked list (or slice) of entries. Simple, robust, slightly more memory.
- Open addressing — linear probing: on collision, walk to the next slot. Cache-friendly. Suffers from clustering.
- Open addressing — quadratic probing: jump by 1, 4, 9, 16, ... reduces clustering.
- Open addressing — double hashing: use a second hash to determine the step.
- Robin Hood hashing: open addressing with displacement balancing — used by Go's modern map.
- Cuckoo hashing: two hash functions, two tables; guarantees O(1) lookup, more complex insert.
- Allocate a new, larger backing array (typically 2× size).
- Re-hash every existing entry into the new array (since the bucket count changed).
- Free or replace the old array.
- Cost: O(n) one-time, but amortized over many inserts → still O(1) per insert on average.
- Incremental resize: Go's real map spreads the rehash work across many subsequent operations to avoid pause spikes.
map is much more sophisticated than the hand-rolled version:
- Compact buckets: 8 entries per bucket, packed for cache locality.
- Top-hash optimization: stores the high byte of each key's hash for fast filtering before full key comparison.
- Incremental rehashing: avoids stop-the-world pauses on large maps.
- Random seed: the hash function is seeded per program to prevent hash-flooding DoS attacks.
- Specialized fast paths for common key types like
intandstring. - Deliberately not concurrent-safe — runtime panics on concurrent writes.
- Why
mapiteration order is unstable (it's randomized to surface order-dependency bugs). - Why bad hash functions destroy performance (everything ends up in one bucket).
- What "amortized O(1)" actually means (most ops are fast, occasional resize is expensive).
- Why
mapcan't be used concurrently (resize while another goroutine is reading = chaos). - How to design custom hash structures for niche use cases (perfect hashing, content-addressed stores).
// Go maps ARE hash tables. This is for understanding.
const tableSize = 64 // number of buckets
type entry struct { key string; value int; next *entry } // linked list node
type HashTable struct { buckets [tableSize]*entry; size int } // fixed array of chains
func hash(key string) int {
h := 0 // running hash value
for _, ch := range key { h = (h*31 + int(ch)) % tableSize } // polynomial rolling hash
return h // bucket index
}
func (ht *HashTable) Put(key string, val int) {
idx := hash(key) // compute bucket
for curr := ht.buckets[idx]; curr != nil; curr = curr.next {
if curr.key == key { curr.value = val; return } // update existing
}
ht.buckets[idx] = &entry{key, val, ht.buckets[idx]} // prepend to chain
ht.size++
}
func (ht *HashTable) Get(key string) (int, bool) {
for curr := ht.buckets[hash(key)]; curr != nil; curr = curr.next { // walk chain
if curr.key == key { return curr.value, true } // found
}
return 0, false // not found
}
map)- Just use
map[K]V— Go's built-in map is a highly optimized hash table; rolling your own is purely educational - Set type —
map[string]struct{}uses zero memory for the value - Frequency counting —
counts := map[string]int{}; counts[word]++ - Caches — request-level memoization, lookup tables for small datasets
- Custom hash for non-comparable keys — Go's map keys must be comparable (no slices/maps); for slice keys, hash to a string first
- Concurrent maps —
sync.Mapfor read-mostly, otherwisemap+sync.RWMutex
49 Sorting
- Type-specific helpers:
sort.Ints,sort.Strings,sort.Float64s— sort built-in types in place. - Closure-based (Go 1.8+):
sort.Slice(s, less)takes a slice and a comparator closure. Less verbose than implementing an interface. - Generic (Go 1.21+):
slices.Sort(s)andslices.SortFunc(s, cmp)— type-safe, fast, the modern preferred way. - The original
sort.Interface: implementLen,Less,Swapfor full control. Still useful for non-slice types.
sort.SliceStable, slices.SortStableFunc) and binary search helpers (sort.Search, slices.BinarySearch).
- Pdqsort (pattern-defeating quicksort) — Go 1.19+ uses this for unstable sorts. Adversarial-input safe, very fast in practice on real data.
- Insertion sort — used for tiny slices (typically <12 elements).
- Heap sort — fallback when pdqsort detects an adversarial pattern.
- Mergesort — used for stable sorts; preserves the relative order of equal elements.
- All sorts are O(n log n) worst case, with O(n) best case for nearly-sorted input.
slices.Sort([]int{3, 1, 4, 1, 5}) // sort in place
slices.SortFunc(users, func(a, b User) int {
return strings.Compare(a.Name, b.Name)
})
slices.SortStableFunc(items, cmpFn) // preserve equal-key order
i, ok := slices.BinarySearch(sortedSlice, x) // binary search
The cmp package provides cmp.Compare and cmp.Less helpers for comparing ordered values.
- vs Python
list.sort(key=...): Python uses a key function that extracts a sort key from each element. Go uses a compare function that takes two elements and returns -1/0/1. Slightly different mental model. - vs Java
Collections.sort: Same shape (compare function); Java usesComparatorinterface or lambda. Go uses a closure or interface. - vs JavaScript
arr.sort(): JS sort is famously stable since ES2019; Go's stable sort is opt-in. - vs C++
std::sort: Both are introsort/pdqsort variants. Same speed, different syntax. - The
sort.Interfacewas once the only way — required for sorting custom collections. Now mostly legacy.
sort.Slice (still everywhere in legacy Go code) and the modern generic slices.SortFunc (in any Go ≥ 1.21) lets you read and write idiomatic code in any codebase. The built-in sort is fast enough that hand-rolling is almost never worth it.- Unstable (default): equal elements may be reordered. Faster.
- Stable: elements with equal keys keep their original order. Slightly slower, but essential for multi-key sorts (sort by date, then by name within date).
- Multi-key sorts are easier with stable sort: sort by least significant key first, then most significant, the result respects all keys.
The sort package provides sorting for built-in types and custom sorting via the sort.Interface or sort.Slice.
import "sort"
// Built-in sorts
numbers := []int{5, 3, 4, 1, 2}
sort.Ints(numbers) // [1, 2, 3, 4, 5]
names := []string{"John", "Anthony", "Steve"}
sort.Strings(names) // ["Anthony", "John", "Steve"]
// Custom sort with sort.Slice (most common)
type Person struct { Name string; Age int }
people := []Person{
{"Alice", 30}, {"Bob", 25}, {"Anna", 35},
}
// Sort by age ascending
sort.Slice(people, func(i, j int) bool {
return people[i].Age < people[j].Age
})
// [{Bob 25} {Alice 30} {Anna 35}]
// Sort by name
sort.Slice(people, func(i, j int) bool {
return people[i].Name < people[j].Name
})
// Sort by last character of string
fruits := []string{"banana", "apple", "cherry"}
sort.Slice(fruits, func(i, j int) bool {
return fruits[i][len(fruits[i])-1] < fruits[j][len(fruits[j])-1]
})
Custom Sort with sort.Interface
// Implement Len(), Less(), Swap() for full control
type By func(p1, p2 *Person) bool
type personSorter struct {
people []Person
by func(p1, p2 *Person) bool
}
func (s *personSorter) Len() int { return len(s.people) }
func (s *personSorter) Less(i, j int) bool { return s.by(&s.people[i], &s.people[j]) }
func (s *personSorter) Swap(i, j int) { s.people[i], s.people[j] = s.people[j], s.people[i] }
func (by By) Sort(people []Person) {
sort.Sort(&personSorter{people, by})
}
// Usage: define sort criteria as functions
ageAsc := func(a, b *Person) bool { return a.Age < b.Age }
By(ageAsc).Sort(people) // sort by age ascending
sort.Sort actually workssort.Sort is a single, generic sorting algorithm (an introsort — quicksort + heapsort fallback) that lives inside the sort package. It was written before generics existed, and it has to sort anything: a slice of ints, a slice of structs, a linked list, even rows in a database cursor. To do that without knowing your data, it needs you to answer three questions about your collection:
Len() int— "How many items are there?" The algorithm needs the bounds so it knows where to start and stop. For a slice this is triviallylen(s.people), but for a linked list or a paged result set the answer is different — only you know how to count your data.Less(i, j int) bool— "Is itemismaller than itemj?" This is the ordering rule. The sort algorithm has no idea whether you want to sort by age, name, or distance from the moon.Lessis the hook where you encode that decision.Swap(i, j int)— "Exchange the items at positionsiandj." The algorithm needs to physically rearrange your data, but it can't touch your slice directly because it doesn't know its element type.Swapis the hook that lets the algorithm move things around without knowing what they are.
Together, these three methods are the contract defined by sort.Interface:
// From the standard library:
type Interface interface {
Len() int
Less(i, j int) bool
Swap(i, j int)
}
func Sort(data Interface) { /* generic algorithm uses the 3 methods */ }
Anything that implements those three methods can be passed to sort.Sort. The algorithm calls Len() once to learn the size, then repeatedly asks Less(i, j) to compare positions and Swap(i, j) to reorder them — and that's literally all it needs.
Why the wrapper struct (personSorter)? Because sort.Interface is implemented on a type, not on a slice literal. You can't directly attach methods to []Person from outside its package, so you wrap the slice (and the comparison function) in a struct and hang the methods off that. The by field is what makes the same wrapper reusable for different sort orders — change by and you sort by name instead of age, with zero new types.
Do you really have to write all three every time? No — sort.Sort is the low-level / "full control" path. In modern Go you almost always use the shortcut:
// One-liner — no struct, no Len/Less/Swap, no wrapper
sort.Slice(people, func(i, j int) bool {
return people[i].Age < people[j].Age
})
// Or with generics (Go 1.21+):
slices.SortFunc(people, func(a, b Person) int { return a.Age - b.Age })
Under the hood, sort.Slice still does the same thing — it builds a hidden adapter that implements Len/Less/Swap for you using reflection on the slice. The three-method dance is the foundation everything else is built on.
When you'd still hand-roll the three methods:
- You're sorting something that isn't a slice (a database cursor, a tree's flattened view, a custom container).
- You want a named, reusable sorter type — e.g.,
ByAge,ByName— exported from a package. - You need
sort.Stableon a non-slice type, or you're writing a library that takessort.Interfaceas a parameter. - Performance:
sort.Sliceuses reflection internally and is measurably slower than a hand-writtensort.Interfacein tight loops.
For 95% of day-to-day code, reach for sort.Slice or slices.SortFunc. Understand the three-method version because it explains how Go's sort works and shows up in older codebases and library APIs.
- Leaderboards — sort users by score before serializing to JSON
- Pagination with stable order —
sort.SliceStablepreserves equal-key ordering for consistent paged results - Multi-key sort — sort by department, then by salary descending:
if a.Dept != b.Dept { return a.Dept < b.Dept }; return a.Salary > b.Salary - Top-K elements — sort and take first K, OR use
container/heapfor O(n log k) - slices.SortFunc (Go 1.21+) — generic and faster than
sort.Slice(no reflection) - Pre-sorted indexes — for repeated lookups, sort once + binary search beats linear scan
50 I/O & Files
io package that everything in the standard library implements:
io.Reader— one method:Read(p []byte) (n int, err error). Reads up tolen(p)bytes intop; returns the count and an error (io.EOFat end of stream).io.Writer— one method:Write(p []byte) (n int, err error). Writeslen(p)bytes fromp; returns count and error.
os.File), network connections (net.Conn), HTTP bodies (http.Request.Body), in-memory buffers (bytes.Buffer, strings.Reader), compressors (gzip.Writer), encoders (json.Encoder) — they all implement these interfaces, so anything that reads/writes data composes with everything else.
- Read entire file:
data, err := os.ReadFile("path")→[]byte. Simple, but reads everything into memory. - Write entire file:
err := os.WriteFile("path", data, 0644)— the second arg is permissions. - Open for reading:
f, err := os.Open("path"); defer f.Close() - Open for writing/append/create:
os.OpenFile("path", os.O_WRONLY|os.O_CREATE|os.O_APPEND, 0644) - Stream a file line-by-line:
scanner := bufio.NewScanner(f); for scanner.Scan() { line := scanner.Text() } - Copy from one stream to another:
n, err := io.Copy(dst, src)— handles all the buffer management. - File info:
fi, _ := os.Stat("path"); fi.Size(); fi.Mode(); fi.ModTime() - Delete:
os.Remove("path");os.RemoveAll("dir")for recursive.
io.Reader/Writer, you can chain operations effortlessly:
- HTTP body → JSON decode:
json.NewDecoder(resp.Body).Decode(&v) - File → gzip → tar:
tw := tar.NewWriter(gzip.NewWriter(file)) - Read with progress bar: wrap a Reader with a progress-counting Reader.
- Hash a stream:
h := sha256.New(); io.Copy(h, file); h.Sum(nil) - Multiple sinks:
io.MultiWriter(file, os.Stdout)writes to both at once. - Multiple sources:
io.MultiReader(r1, r2, r3)concatenates streams.
- vs Java: Java has
InputStream/OutputStream+Reader/Writer(separate for binary vs text) plus dozens of decorator subclasses. Go has just two interfaces and they're text-or-binary at your discretion. - vs Python: Python's file objects have many methods; Go's interfaces are minimal and composable.
- vs Node.js streams: Node has
Readable/Writable/Duplex/Transformwith event-based and async iterator APIs. Powerful but complex; Go's is simpler. - vs C
FILE*: Untyped, error-prone, no abstraction. Go's interfaces are dramatically cleaner. - The whole stdlib follows the convention:
encoding/json,encoding/xml,compress/gzip,net/http,image/png,crypto/cipher— all useio.Reader/io.Writer.
io.Reader/io.Writer instead of *os.File or []byte makes them trivially testable (pass a strings.Reader in tests, a real file in prod, an HTTP body in production). It also makes them instantly composable with the entire Go ecosystem. This single design decision is one of the cleanest and most copied parts of Go's stdlib.- Always
defer file.Close()immediately after Open. - Don't
os.ReadFileon huge files — use a buffered scanner or io.Copy. - Prefer
io.Copyfor stream-to-stream — it picks the right buffer size and handles edge cases. - Take
io.Readerin function signatures, not*os.File, so callers can pass anything. - Wrap with
bufiofor line-oriented or byte-by-byte reads to avoid syscall overhead.
// Read entire file
data, err := os.ReadFile("config.json") // returns []byte
// Write entire file
os.WriteFile("out.txt", []byte("hello"), 0644) // 0644 = rw-r--r-- perms
// Read line by line
f, _ := os.Open("file.txt") // open read-only
defer f.Close() // always close the file
scanner := bufio.NewScanner(f) // wraps file in a scanner
for scanner.Scan() { fmt.Println(scanner.Text()) } // Text() returns current line
// Buffered write
f, _ = os.Create("out.txt") // create or truncate file
w := bufio.NewWriter(f) // buffer writes for efficiency
w.WriteString("line 1\n") // write to buffer
w.Flush() // don't forget!
// io.Copy between reader and writer
io.Copy(dst, src) // stream bytes from src to dst
// Directory operations
os.MkdirAll("path/to/dir", 0755) // create all parent dirs too
entries, _ := os.ReadDir(".") // list current directory
_ = data; _ = err; _ = entries // suppress unused-var errors
- io.Reader/io.Writer everywhere — files, network sockets, gzip streams, S3 downloads — all share the same interface
- io.Copy for large files — streams in chunks instead of loading into memory;
io.Copy(dst, src)handles gigabyte files - Don't ReadFile huge files —
os.ReadFileloads everything into memory; use bufio.Scanner for line-by-line processing - Always defer Close — open file → check error → defer close immediately to prevent leaks
- Atomic writes — write to
file.tmp, thenos.Rename; rename is atomic on POSIX systems - os.OpenFile with flags —
O_APPEND|O_CREATE|O_WRONLYfor log files;O_EXCLfor "create if not exists"
51 Bufio & Line Filters
bufio package — in-memory buffers wrapped around any io.Reader/io.Writer to amortize the cost of small reads and writes. The package provides three main types:
bufio.Reader— wraps anio.Readerwith a fixed-size internal buffer (default 4 KB). Reads from the wrapped Reader in chunks, hands out bytes from the buffer. Methods:ReadByte,ReadRune,ReadString(delim),ReadBytes(delim),Peek(n).bufio.Writer— wraps anio.Writer; small writes accumulate in a buffer until youFlush(or the buffer fills). Drastically reduces syscall count.bufio.Scanner— higher-level helper for token-by-token reading. Default split mode is by lines, but you can also split by words, runes, or a custom function.
file.Read([]byte{0}) can trigger a system call (a switch into kernel mode). Without buffering:
- Reading a 100 MB file one byte at a time → 100 million syscalls.
- Each syscall costs ~500 ns to a few microseconds → tens of seconds wasted.
- With a 4 KB buffer, you do ~25,000 syscalls instead — 1000× fewer.
scanner := bufio.NewScanner(file)
for scanner.Scan() {
line := scanner.Text() // current line, no trailing \n
process(line)
}
if err := scanner.Err(); err != nil { /* handle */ }
- Default split: by lines.
- Other split functions:
scanner.Split(bufio.ScanWords),ScanRunes,ScanBytes, or a customSplitFunc. - Default max token size: 64 KB. For long lines, call
scanner.Buffer(buf, max)to allow larger. - Always check
scanner.Err()after the loop — silent EOF vs real error matters.
- vs Java
BufferedReader: Same idea, same name. Java'sreadLine()equivalent is Go'sbufio.Scanner. - vs Python
io.BufferedReader: Python wraps file objects with buffering by default; Go requires you to opt in withbufio.NewReader. - vs C
FILE*: C streams are buffered by default. Go is more explicit — rawos.Fileis unbuffered. - vs
bufio.Reader.ReadString: alternative to Scanner — keeps the delimiter in the result, doesn't have the 64 KB cap, but slightly more verbose.
bufio.Scanner is the standard line-by-line reader for log processors, ETL scripts, CLI filters, parsers, and any other "read text input one logical unit at a time" task. bufio.Writer is critical for high-throughput log writers, code generators, and CSV writers — anywhere you'd otherwise be hammering disk with tiny writes.- Always
defer w.Flush()after creating abufio.Writer— otherwise your last buffered writes silently disappear. - Scanner's 64 KB default cap bites on long lines (huge JSON logs, base64 binaries). Use
scanner.Buffer(buf, 1024*1024). - Don't reuse a Scanner after it returns false — create a new one.
- Scanner silently swallows errors on huge tokens — always check
scanner.Err(). - Wrapping an already-buffered reader is wasteful — don't wrap
bufio.Readerin anotherbufio.Reader.
The bufio package wraps io.Reader/io.Writer with a buffer for efficient I/O. Essential for reading files line-by-line or writing many small pieces.
import "bufio"
// Buffered Writer — batches small writes
writer := bufio.NewWriter(os.Stdout)
writer.Write([]byte("Hello, bufio!\n")) // writes to buffer, not stdout yet
writer.WriteString("Direct string\n") // also buffered
writer.Flush() // FLUSH sends buffer to actual writer. Don't forget!
// Buffered Reader — efficient reads
reader := bufio.NewReader(file)
data := make([]byte, 20)
n, _ := reader.Read(data) // read up to 20 bytes
line, _ := reader.ReadString('\n') // read until newline
// Scanner — most common way to read line-by-line
scanner := bufio.NewScanner(file)
for scanner.Scan() {
line := scanner.Text() // current line (without \n)
fmt.Println("Line:", line)
}
if err := scanner.Err(); err != nil {
fmt.Println("Error:", err)
}
Line Filter Example
// Read a file, filter lines containing a keyword, replace text
file, _ := os.Open("example.txt")
defer file.Close()
scanner := bufio.NewScanner(file)
keyword := "important"
for scanner.Scan() {
line := scanner.Text()
if strings.Contains(line, keyword) {
updated := strings.ReplaceAll(line, "important", "necessary")
fmt.Println("Original:", line)
fmt.Println("Updated:", updated)
}
}
- Parsing log files —
bufio.Scannerfor line-by-line iteration; default 64KB line limit (usescanner.Bufferfor longer) - CSV/JSONL processing — stream-process gigabyte files without loading them entirely into memory
- Network protocols —
bufio.Readerwraps a TCPnet.Connfor efficient line/delimited reads - Stdin parsing —
bufio.NewScanner(os.Stdin)for interactive CLI tools - Buffered writers for performance — wrap an
os.Fileinbufio.NewWriterfor many small writes; ALWAYS callFlush() - Custom split functions —
scanner.Split(bufio.ScanWords)for word tokenization; write your own for protocols
52 File Paths & Directories
path/filepath— handles OS-specific file paths. Uses/on Unix,\on Windows. The right tool for working with the actual filesystem.path— handles forward-slash-only paths used by URLs, embedded filesystems (embed.FS), and other contexts where/is universal regardless of OS.
filepath for disk paths, path for URL paths.
Join(parts...)— combine path parts with the OS separator; also cleans the result.Split(p)— split into(dir, file).Base(p)— last element of the path (the filename).Dir(p)— everything except the last element.Ext(p)— file extension including the dot (.txt).Abs(p)— convert to absolute path.Rel(base, target)— compute target relative to base.Clean(p)— normalize: collapse.., remove redundant slashes.Glob(pattern)— match filenames using shell-style wildcards (*,?,[abc]).Walk(root, fn)— traverse a directory tree, callingfnfor every entry.WalkDir(root, fn)(Go 1.16+) — faster modern replacement for Walk; usesDirEntryinstead ofFileInfo.Match(pattern, name)— test a single name against a glob pattern.
- Read directory:
entries, _ := os.ReadDir("path")— modern API returning[]DirEntry. - Create directory:
os.Mkdir("path", 0755) - Create directory tree:
os.MkdirAll("a/b/c", 0755) - Remove:
os.Remove(path)— single file/empty dir;os.RemoveAll(path)— recursive. - Stat:
fi, _ := os.Stat(path); fi.Size(); fi.IsDir(); fi.Mode() - Rename / move:
os.Rename(old, new) - Working directory:
os.Getwd(),os.Chdir(dir) - Home directory:
os.UserHomeDir()
- vs hand-concatenating strings:
dir + "/" + filebreaks on Windows.filepath.Joinuses the correct separator ANDCleans the result. - vs Python
os.path/pathlib: Same idea. Python'spathlib.Pathis more object-oriented; Go's API is functional. - vs Java
java.nio.file.Path: Comparable feature set. Go is simpler. - The two-package split is unique to Go and exists because URL paths and disk paths have different separator conventions.
filepath from the start guarantees portability with no extra effort. filepath.WalkDir + filepath.Glob are the foundation of any tool that processes file trees: linters, formatters, build systems, deployment scripts, log processors.import "path/filepath"
// Join paths (OS-aware separators)
p := filepath.Join("home", "Documents", "file.zip")
// Linux: "home/Documents/file.zip"
// Windows: "home\Documents\file.zip"
// Split into directory and file
dir, file := filepath.Split("/home/user/docs/file.txt")
// dir: "/home/user/docs/" file: "file.txt"
// Get parts
filepath.Base("/home/user/docs/") // "docs"
filepath.Ext("file.txt") // ".txt"
filepath.IsAbs("/home/user") // true
filepath.IsAbs("./data") // false
// Normalize messy paths
filepath.Clean("./data/../data/file.txt") // "data/file.txt"
// Relative path between two paths
rel, _ := filepath.Rel("a/b", "a/b/t/file") // "t/file"
// Convert relative to absolute
abs, _ := filepath.Abs("./data/file.txt")
// Remove extension from filename
name := strings.TrimSuffix("file.txt", filepath.Ext("file.txt")) // "file"
Directory Operations
// Create directories
os.Mkdir("subdir", 0755) // single directory
os.MkdirAll("subdir/parent/child", 0755) // create all parents too
// List directory contents
entries, _ := os.ReadDir("subdir")
for _, entry := range entries {
fmt.Println(entry.Name(), entry.IsDir())
}
// Walk entire directory tree recursively
filepath.WalkDir("subdir", func(path string, d fs.DirEntry, err error) error {
fmt.Println(path)
return nil
})
// Get/change working directory
dir, _ := os.Getwd() // current working directory
os.Chdir("subdir") // change directory
// Remove
os.Remove("file.txt") // single file/empty dir
os.RemoveAll("subdir") // recursive delete (like rm -rf)
- filepath.Join over string concat — handles trailing slashes, OS-specific separators, relative paths correctly
- filepath.Walk for find-like tools — implementing
find,du, file scanners;filepath.SkipDirprunes branches - filepath.Clean for security — sanitize user-supplied paths; check for
..traversal attacks - Use slash-separated for URLs —
pathpackage (notpath/filepath) for URL paths — always uses/ - os.UserConfigDir / os.UserCacheDir — XDG-compliant locations for app config and cache files
- fs.FS interface — Go 1.16+ abstraction over filesystems; works with
embed.FS, real disk, in-memory FS for tests
53 Temp Files & Directories
os package for creating uniquely-named temporary files and directories:
os.CreateTemp(dir, pattern)— creates and opens a new temporary file.diris the parent directory (""= OS default temp).patternis the filename template; if it contains*, the random part replaces it; otherwise the random part is appended.os.MkdirTemp(dir, pattern)— creates a new temporary directory with a unique random name. Returns the path.
O_EXCL creation so the name is guaranteed unique even under concurrent use. Both return the actual path (with the random suffix), which you must clean up.
- Linux:
/tmp(or$TMPDIRif set). - macOS:
$TMPDIR(typically a long random path under/var/folders/...). - Windows:
%TEMP%(typicallyC:\Users\<name>\AppData\Local\Temp). - Get the location with
os.TempDir(). - Override with
$TMPDIRin the environment.
os.Create("/tmp/mydata.txt") has multiple problems:
- Race condition: another process or another goroutine could create the same file between the time you check and the time you create.
- Symlink attacks: a malicious user could pre-create the file as a symlink to
/etc/passwdor similar. - Multi-instance conflicts: two copies of your tool running at the same time will collide.
- Predictable names in shared temp directories are a classic security vulnerability.
CreateTemp uses O_CREAT | O_EXCL under the hood, atomically creating only if the file doesn't exist — guaranteed unique.
- vs Python
tempfile.NamedTemporaryFile: Same idea. Python's auto-deletes on context exit; Go requiresdefer os.Removemanually. - vs Java
Files.createTempFile: Same approach, slightly different naming pattern syntax. - vs C
mkstemp: What both Python and Go use under the hood on Unix. - Go 1.16 rename: the old
ioutil.TempFileandioutil.TempDirwere renamed toos.CreateTempandos.MkdirTempas part of theioutildeprecation. Old code still works but new code should use the new names.
- Atomic file writes — write to a temp file, then rename atomically over the target. Prevents readers from seeing a half-written file.
- Test fixtures — create a temp directory, populate it, run the test, clean up. Use
t.TempDir()in tests for automatic cleanup. - Scratch buffers — image processing, video encoding, large data transformations.
- Shelling out to external tools that need a real file path (ffmpeg, imagemagick).
- Download then move patterns — download to temp, validate, then rename.
- Always pair with
defer os.Remove(f.Name())for files ordefer os.RemoveAll(dir)for directories. - In tests, use
t.TempDir()— automatically cleaned up after the test. - Don't forget to
defer f.Close()in addition to the remove. - Pattern with
*placement matters:"prefix-*.txt"puts the random part in the middle.
// Create a temporary file
tempFile, err := os.CreateTemp("", "myprefix") // "" = OS temp dir
if err != nil { panic(err) }
fmt.Println("Temp file:", tempFile.Name()) // e.g., /tmp/myprefix123456
defer os.Remove(tempFile.Name()) // clean up when done
defer tempFile.Close()
// Create a temporary directory
tempDir, err := os.MkdirTemp("", "GoCourseTempDir")
if err != nil { panic(err) }
fmt.Println("Temp dir:", tempDir)
defer os.RemoveAll(tempDir) // clean up recursively
// Use case: write temp data, process it, then clean up
// The "" first arg means use the system's default temp directory
// You can pass a specific directory path instead
- Atomic file replacement — write to a temp file in the same directory, then
os.Renameto the final name (atomic on POSIX) - Test isolation —
t.TempDir()creates a per-test directory auto-cleaned after test runs - Image/video processing — extract uploads to temp files for processing pipelines (ffmpeg, imagemagick) that need real filesystem paths
- Build artifacts — Go's own toolchain uses temp dirs for compilation intermediates
- Always defer Remove — temp files leak; in long-running processes they fill
/tmpover weeks - Predictable locations — pass a specific dir as first arg for sandboxes that don't allow
/tmpaccess
54 JSON & Encoding
encoding/json standard-library package — Go's full implementation of JSON encoding and decoding. Two main entry points:
json.Marshal(v)— turns a Go value into JSON bytes ([]byte).json.Unmarshal(data, &v)— parses JSON bytes into a Go value (typically a struct ormap[string]any).
json.NewEncoder(w).Encode(v)— write JSON directly to anio.Writer. Used in HTTP handlers:json.NewEncoder(w).Encode(response).json.NewDecoder(r).Decode(&v)— parse JSON from anio.Reader. Used in HTTP handlers:json.NewDecoder(r.Body).Decode(&req).json.MarshalIndent(v, "", " ")— pretty-printed output for human readers.
json:"name,omitempty" struct tags.
json:"name"— use this name in JSON instead of the Go field name.json:"name,omitempty"— omit the field if it's the zero value (0, "", nil, false).json:"-"— never include this field, even if exported.json:"name,string"— encode/decode a numeric field as a JSON string (handy for big numbers in JS clients).json:",inline"(3rd-party only; stdlib uses field embedding for this).- No tag = the field name is used as-is, but only for exported (capitalized) fields.
- Go bool ↔ JSON bool
- Go numeric (int, float) ↔ JSON number
- Go string ↔ JSON string
- Go slice ↔ JSON array (nil slice →
null; empty slice →[]) - Go map[string]X ↔ JSON object
- Go struct ↔ JSON object (using exported fields and tags)
- Go interface{} / any ↔ any JSON type (decoded as
bool,float64,string,[]any, ormap[string]any) - Go time.Time ↔ JSON RFC 3339 string
- Custom marshaling: implement
MarshalJSON()/UnmarshalJSON()for full control.
- vs Python: Python uses runtime dicts (
json.loadsreturns adict). Go decodes into typed structs by default — far safer. - vs Java/Kotlin: Java needs Jackson, Gson, or kotlinx.serialization with annotations. Go ships JSON in stdlib with struct tags — no extra dependency.
- vs Node.js: Node has
JSON.parse/JSON.stringifybut no type checking. Go gives you typed access plus the option of dynamic viaany. - vs Rust serde: Serde is more powerful (zero-copy, attribute-rich) but more complex. Go's API is simpler.
- Reflection-based: Go's encoder uses
reflectinternally, which is convenient but slower than typed encoders likejson-iterator/goor code-generatedeasyjson/ffjson. For hot paths, those alternatives are 3–10× faster. - Field visibility follows Go's exported rule: only capitalized fields end up in JSON, regardless of tags.
encoding/json is "good enough" for almost everything: REST APIs, microservice communication, config files, log structures, persistence layers. It integrates seamlessly with net/http handlers, and the struct-tag system keeps your data shape and your wire format in one place. The streaming Decoder is critical for large request bodies — it doesn't load the whole thing into memory.- Decoding into
anyturns numbers intofloat64always — usejson.Numberif you need precise integer handling. - Unknown fields silently ignored by default. Use
decoder.DisallowUnknownFields()for strict APIs. - Maps with non-string keys are not supported in standard JSON; they marshal to objects with stringified keys.
- nil slice marshals to
null, but empty slice marshals to[]. Often matters for clients. - Time zone:
time.Timemarshals as RFC 3339; the time zone is preserved. - Decoding into a pre-populated struct only overwrites fields present in the JSON — defaults stay.
type User struct {
ID int `json:"id"` // maps Go field to JSON key
Name string `json:"name"`
Email string `json:"email,omitempty"` // omit if empty string
Pass string `json:"-"` // never in JSON
}
// Struct → JSON
jsonBytes, _ := json.Marshal(user) // compact JSON bytes
jsonBytes, _ = json.MarshalIndent(user, "", " ") // pretty-printed JSON
// JSON → Struct
var u User // target struct to populate
json.Unmarshal([]byte(jsonStr), &u) // must pass pointer
// JSON → dynamic map
var data map[string]any // use when structure is unknown
json.Unmarshal([]byte(jsonStr), &data) // parse into map
// Streaming (for HTTP bodies)
json.NewEncoder(w).Encode(user) // write to io.Writer
json.NewDecoder(r.Body).Decode(&u) // read from io.Reader
_ = jsonBytes; _ = data // suppress unused-var errors
- REST API request/response — most common use; Marshal for output, Unmarshal for input
- NewDecoder over Unmarshal for HTTP — streaming, doesn't buffer entire body; better for large requests
- RawMessage for delayed parsing — store nested JSON as
json.RawMessageand parse lazily based on a discriminator field - Custom MarshalJSON — implement on a type to control output format (e.g., format dates as ISO strings)
- Disallow unknown fields —
dec.DisallowUnknownFields()rejects payloads with extra keys (useful for strict APIs) - encoding/json/v2 (Go 1.25+) — faster, more correct version with stricter defaults
- map[string]any for unknown shapes — third-party webhooks where you don't control the schema
55 XML Encoding
encoding/xml standard-library package — Go's full implementation of XML encoding and decoding, with the same shape as encoding/json:
xml.Marshal(v)— Go value → XML bytes.xml.Unmarshal(data, &v)— XML bytes → Go value.xml.NewEncoder(w)/xml.NewDecoder(r)— streaming variants.xml.MarshalIndent(v, "", " ")— pretty-printed output.
xml:"name"— element name.xml:"name,attr"— encode as an attribute on the parent element instead of a child element.xml:"name,chardata"— content of the element as character data.xml:"name,cdata"— wrap content in<![CDATA[...]]>.xml:"name,innerxml"— raw inner XML preserved as-is.xml:"name,comment"— write as an XML comment.xml:"a>b>c"— nest the field under multiple parent elements.xml:"-"— exclude.xml:",omitempty"— same as JSON: skip zero values.XMLName xml.Name `xml:"root"`— controls the outer element name.
- vs Java JAXB: JAXB is heavy and annotation-driven. Go's
encoding/xmlis far smaller and stays close to the data model. - vs Python
xml.etree/lxml: Python's APIs are tree-based (DOM-like). Go decodes directly into structs, more like data binding. - vs
encoding/json: Same shape API but with extra tag features for XML's richer model. Attributes vs elements is the biggest difference from JSON. - No schema validation (DTD/XSD): the package can parse XML but won't validate against a schema. For schema validation, use
libxml2bindings or a third-party library. - Namespaces are supported via
xml.Name{Space: "...", Local: "..."}, but they're more verbose than JSON has any equivalent for.
- Finance: SWIFT messages, FpML, FIX.
- Healthcare: HL7, FHIR, CDA documents.
- Government: XBRL filings, GovTalk messages.
- Enterprise SOAP services still common in legacy integration.
- SVG and RSS/Atom feeds.
- Office formats: docx, xlsx, pptx are all zipped XML.
- Build systems: Maven POM, Ant, MSBuild.
- XML output isn't self-prefixed — you need to write
xml.Headermanually for the<?xml ... ?>declaration. - Mixed content (text inside elements with children) is tricky — use
InnerXML. - Namespaces require care — use
xml:"name http://example.com"tag to declare a namespace. - Large XML files: use the streaming Decoder +
Token()for SAX-style parsing instead of loading everything.
Works exactly like JSON encoding but uses encoding/xml. Struct tags control element names, attributes, and omission.
import "encoding/xml"
type Person struct {
XMLName xml.Name `xml:"person"` // root element name
Name string `xml:"name"`
Age int `xml:"age,omitempty"` // omit if zero
Email string `xml:"-"` // always exclude
Address Address `xml:"address"` // nested element
}
// Marshal struct → XML
person := Person{Name: "John", Address: Address{City: "Oakland", State: "CA"}}
xmlData, _ := xml.MarshalIndent(person, "", " ")
fmt.Println(string(xmlData))
// <person>
// <name>John</name>
// <address>
// <city>Oakland</city>
// <state>CA</state>
// </address>
// </person>
// Unmarshal XML → struct
raw := `<person><name>Jane</name><age>25</age></person>`
var p Person
xml.Unmarshal([]byte(raw), &p)
// XML attributes with ,attr tag
type Book struct {
XMLName xml.Name `xml:"book"`
ISBN string `xml:"isbn,attr"` // becomes attribute, not element
Title string `xml:"title,attr"` // <book isbn="..." title="...">
}
- SOAP APIs — legacy enterprise services (banking, telecoms) still use SOAP envelopes
- RSS / Atom feeds — blog feeds, podcast feeds use XML;
encoding/xmlhandles them well - Sitemaps & RSS — generating XML sitemaps for search engine indexing
- Maven / NuGet manifests — build tools that consume
pom.xml,.csprojfiles - Office documents — DOCX/XLSX are zipped XML; libraries like
excelizewrap this - Prefer JSON for new APIs — XML is verbose, slower to parse, harder to debug
56 Base64 Encoding
+ and /, or - and _ for URL-safe). Every 3 input bytes become 4 output characters, with = padding at the end if the input length isn't a multiple of 3. Result: a ~33% size increase.Go's
encoding/base64 package ships four pre-defined encodings:
StdEncoding— standard RFC 4648, uses+//, with padding.URLEncoding— URL-safe variant, uses-/_, with padding.RawStdEncoding— standard without padding.RawURLEncoding— URL-safe without padding (used in JWT headers/payloads).
- Encode bytes to string:
s := base64.StdEncoding.EncodeToString(data) - Decode string to bytes:
b, err := base64.StdEncoding.DecodeString(s) - Encode to writer:
w := base64.NewEncoder(base64.StdEncoding, dst); w.Write(data); w.Close() - Decode from reader:
r := base64.NewDecoder(base64.StdEncoding, src) - Custom alphabet:
base64.NewEncoding("...64chars...").
- vs encryption: Base64 is NOT encryption — anyone can decode it instantly. It hides nothing.
- vs compression: Base64 makes data larger, not smaller (~33% bigger).
- vs hex (base16): Hex is 2× the size; base64 is ~1.33×. Hex is more human-readable; base64 is more compact.
- vs base32 / base58: base32 is bigger but case-insensitive; base58 (Bitcoin addresses) avoids look-alike characters.
- Standard vs URL variants are NOT cross-decoder-compatible: a string encoded with
StdEncodingwon't decode withURLEncodingif it contains+or/. Mixing them is a common bug.
- JWT tokens — header and payload are URL-safe base64-encoded JSON.
- Inline images in HTML/CSS:
data:image/png;base64,iVBORw0KGgo... - HTTP Basic auth:
Authorization: Basic dXNlcjpwYXNz(base64 ofuser:pass). - Email attachments — MIME encodes binaries as base64.
- Embedding binary in JSON/XML — JSON has no native binary type.
- API tokens / random IDs — encoding random bytes from
crypto/randas URL-safe base64 gives compact, copy-pasteable tokens. - QR code payloads, document signatures, WebSocket frames, etc.
- Don't use base64 for security — it provides zero confidentiality. Always combine with encryption when secrecy matters.
- Padding (
=) is sometimes stripped in URL contexts; useRawURLEncoding. - Whitespace in input — standard base64 is strict; some libraries forgive newlines, Go's by default does not.
- Non-canonical input — extra padding or wrong characters cause decode errors. Validate input.
Base64 converts binary data to ASCII text. Used for embedding images, transmitting binary in JSON/XML, and basic data encoding.
import "encoding/base64"
data := []byte("Hello, Base64 Encoding")
// Encode to Base64
encoded := base64.StdEncoding.EncodeToString(data)
fmt.Println(encoded) // "SGVsbG8sIEJhc2U2NCBFbmNvZGluZw=="
// Decode from Base64
decoded, err := base64.StdEncoding.DecodeString(encoded)
if err != nil { fmt.Println("Error:", err); return }
fmt.Println(string(decoded)) // "Hello, Base64 Encoding"
// URL-safe encoding (avoids / and + which break URLs)
urlSafe := base64.URLEncoding.EncodeToString(data)
fmt.Println("URL Safe:", urlSafe)
// StdEncoding: uses + and / (standard)
// URLEncoding: uses - and _ (safe for URLs and filenames)
- Embedding binary in JSON — JSON has no native bytes type; encode binary as base64 strings
- Data URIs — embedding small images directly into HTML/CSS:
data:image/png;base64,iVBOR... - HTTP Basic Auth —
Authorization: Basic <base64(user:password)>header - JWT tokens — header.payload.signature each base64-URL-encoded; use
base64.RawURLEncoding(no padding) - API keys / session tokens — random bytes from
crypto/randbase64-encoded for safe transport - Email attachments — MIME uses base64 to encode binaries inside text-only email bodies
57 Hashing & Cryptography
- Hashing:
crypto/md5(broken, legacy only),crypto/sha1(also weak),crypto/sha256,crypto/sha512,crypto/sha3,crypto/blake2b. - HMAC:
crypto/hmac— keyed hash for message authentication. - Symmetric encryption:
crypto/aes,crypto/cipher(block cipher modes: GCM, CTR, CBC). - Asymmetric:
crypto/rsa,crypto/ecdsa,crypto/ed25519. - TLS:
crypto/tls— complete TLS 1.2/1.3 client and server. - X.509:
crypto/x509— certificate parsing and validation. - Random:
crypto/rand— secure RNG. - Extended:
golang.org/x/cryptofor bcrypt, argon2, scrypt, nacl, ed25519, ssh.
- Same input → same hash always.
- Different input → almost certainly different hash (collisions are computationally infeasible).
- Tiny change → completely different hash (avalanche effect).
- Cannot be reversed to recover the input.
h := sha256.New(); h.Write(data); sum := h.Sum(nil).
- The data hasn't been tampered with (integrity).
- The sender knew the secret key (authenticity).
mac := hmac.New(sha256.New, key); mac.Write(data); sum := mac.Sum(nil). Always verify HMACs with hmac.Equal, not bytes.Equal — the constant-time comparison prevents timing attacks.
- bcrypt (
golang.org/x/crypto/bcrypt) — battle-tested, simple API, tunable cost. - argon2 (
golang.org/x/crypto/argon2) — modern winner of the password hashing competition. - scrypt (
golang.org/x/crypto/scrypt) — older but solid.
- vs OpenSSL bindings: Go has pure-Go crypto — no C dependency, no version mismatch risk, easier cross-compilation.
- vs Java JCE: Java's JCE was famously restricted (export-grade keys by default until recently); Go has none of that nonsense.
- vs Python: Python has
hashlibandcryptography(third-party). Go has more in stdlib. - vs Node.js
crypto: Comparable scope; Go is more type-safe. - Strong defaults: Go's package layout makes weak primitives obvious —
crypto/md5exists but you'd never type it without thinking.
- SHA-256: content addressing (Git, IPFS, Docker images, integrity checks).
- HMAC: signed cookies, webhook signature verification (Stripe, GitHub), API request signing.
- bcrypt/argon2: password storage.
- AES-GCM: symmetric encryption with built-in authentication.
- RSA/ECDSA/Ed25519: public-key signatures, JWT signing, TLS handshakes.
- TLS: every HTTPS connection.
Hashing creates a fixed-size fingerprint of data. One-way (can't reverse), deterministic (same input = same hash), and collision-resistant.
import (
"crypto/rand"
"crypto/sha256"
"encoding/base64"
"io"
)
// Simple SHA-256 hash
hash := sha256.Sum256([]byte("password123"))
fmt.Printf("SHA-256: %x\n", hash)
// Password hashing with SALT (proper way)
func generateSalt() ([]byte, error) {
salt := make([]byte, 16)
_, err := io.ReadFull(rand.Reader, salt) // crypto-secure random
return salt, err
}
func hashPassword(password string, salt []byte) string {
salted := append(salt, []byte(password)...)
hash := sha256.Sum256(salted)
return base64.StdEncoding.EncodeToString(hash[:])
}
// Sign-up: generate salt, hash password, store both
salt, _ := generateSalt()
signUpHash := hashPassword("password123", salt)
// Store salt + signUpHash in database
// Login: retrieve salt from DB, hash input, compare
loginHash := hashPassword("password123", salt)
if signUpHash == loginHash {
fmt.Println("Login successful!")
}
Use bcrypt or argon2 for real password hashing. SHA-256 is too fast — attackers can brute-force billions of hashes/second. bcrypt is intentionally slow. This example teaches the concept; use golang.org/x/crypto/bcrypt in production.
- File integrity — SHA-256 checksums for verifying downloads (Linux distro ISOs, npm packages)
- Password hashing —
golang.org/x/crypto/bcryptorargon2— NEVER plain SHA - HMAC for API signing —
hmac.New(sha256.New, secret)for webhook verification (Stripe, GitHub, Twilio) - JWT signing — HS256 (HMAC-SHA256) for shared-secret tokens; RS256 (RSA) for public-key
- TLS/HTTPS —
crypto/tlshandles cert verification;crypto/x509for parsing certs - crypto/rand for secrets — never use
math/randfor tokens, salts, IVs - AES encryption —
crypto/aes+crypto/cipherfor symmetric encryption (always use authenticated modes like GCM)
58 URL Parsing
net/url package — Go's RFC 3986-compliant URL parser. The main entry point is url.Parse(rawurl), which takes a URL string and returns a *url.URL struct with separate, typed fields for every URL component. Plus helpers for percent-encoding and query string manipulation.[scheme://][userinfo@]host[:port][/path][?query][#fragment]. After url.Parse:
u.Scheme— "https", "http", "ftp", "mailto", etc.u.User—*url.UserinfowithUsername()andPassword().u.Host— host + optional port (e.g. "example.com:8080" or "[::1]:80" for IPv6).u.Hostname()/u.Port()— split helpers.u.Path— decoded path;u.RawPathpreserves original encoding.u.RawQuery— query string without the leading?; useu.Query()for parsed map.u.Fragment— anchor without the leading#.u.String()— reconstruct the URL.
url.Valuesis amap[string][]string— multiple values per key, like HTML form data.- Parse:
vals := u.Query()orvals, _ := url.ParseQuery(s) - Get first value:
vals.Get("key")— returns "" if absent. - All values:
vals["key"]— slice of strings. - Set/Add:
vals.Set("key", "value")(replaces),vals.Add("key", "value")(appends). - Encode:
vals.Encode()→ URL-encoded query string with sorted keys. - Percent-encoding helpers:
url.QueryEscape(s),url.QueryUnescape(s),url.PathEscape(s),url.PathUnescape(s).
- Percent-encoded characters:
/path%2Fwith%2Fslashes - IPv6 hosts:
https://[::1]:8080/ - Userinfo with special chars:
https://user%40domain:p@ss@host/ - Empty components:
http:///path - Schemes with no host:
mailto:user@example.com,file:///etc/passwd - Default ports omitted from one source vs explicit in another.
url.Parse handles all of it correctly per the RFC.
- vs JavaScript: JS now has
new URL(...)which is comparable; older code used thedocument.createElement('a').hrefhack. - vs Python
urllib.parse: Same idea, similar API. Go is strongly typed. - vs Java
java.net.URL: Java's URL class is famously buggy (does network lookups inequals()!). Go's is purely lexical. - Resolving relative URLs:
base.ResolveReference(ref)handles "page2" relative to "/dir/page1".
import "net/url"
// Parse a URL into components
// [scheme://][userinfo@]host[:port][/path][?query][#fragment]
raw := "https://example.com:8080/path?query=param#fragment"
u, _ := url.Parse(raw)
fmt.Println(u.Scheme) // "https"
fmt.Println(u.Host) // "example.com:8080"
fmt.Println(u.Port()) // "8080"
fmt.Println(u.Path) // "/path"
fmt.Println(u.RawQuery) // "query=param"
fmt.Println(u.Fragment) // "fragment"
// Parse query parameters
u2, _ := url.Parse("https://example.com?name=John&age=30")
params := u2.Query()
fmt.Println(params.Get("name")) // "John"
fmt.Println(params.Get("age")) // "30"
// Build a URL programmatically
baseURL := &url.URL{Scheme: "https", Host: "example.com", Path: "/search"}
q := baseURL.Query()
q.Set("name", "John")
q.Set("city", "London")
baseURL.RawQuery = q.Encode()
fmt.Println(baseURL.String()) // "https://example.com/search?city=London&name=John"
// url.Values — encode query strings manually
values := url.Values{}
values.Add("name", "Jane")
values.Add("age", "30")
encoded := values.Encode() // "age=30&name=Jane"
- Parsing webhook callbacks — extract path/query params from incoming URLs
- Building API requests — programmatically construct URLs with
url.Valuesto avoid manual escaping bugs - OAuth flows — parsing redirect URLs with code/state query params
- URL-encode user input —
url.QueryEscape(searchTerm)before appending to query strings - Validate trusted hosts — parse URL, check
u.Hostagainst an allowlist before redirecting (prevent open-redirect) - Reverse proxies — parse, mutate, re-emit URLs when forwarding requests
59 Text Templates
text/template— for plain text output (config files, code generation, plain emails). No escaping.html/template— for HTML output. Automatically performs context-aware escaping to prevent XSS. Uses the same syntax as text/template.
{{ ... }} for actions: substitution ({{ .Field }}), conditionals, loops, function calls, and variable definitions.
- Field access:
{{ .Name }}— outputs theNamefield of the data passed toExecute. - Conditionals:
{{ if .Active }}...{{ else }}...{{ end }} - Loops:
{{ range .Items }}{{ .Name }}{{ end }} - Range with index:
{{ range $i, $v := .Items }}{{ $i }}: {{ $v }}{{ end }} - Variables:
{{ $name := "world" }}Hello, {{ $name }} - Pipelines:
{{ .Name | upper | printf "%q" }}— chain functions left to right. - With (scoped):
{{ with .User }}{{ .Name }}{{ end }}— sets.to.Userwithin the block. - Sub-templates:
{{ template "name" .Data }}— call a named template. - Define:
{{ define "header" }}...{{ end }}— declare a named template. - Comments:
{{/* this is a comment */}}
- Comparison:
eq,ne,lt,le,gt,ge - Logic:
and,or,not - Output:
print,printf,println - Inspection:
len,index(for maps/slices) - HTML/JS/URL escape functions in
html/template. - Custom functions: register your own with
tmpl.Funcs(template.FuncMap{...}).
html/template: the same data is escaped differently depending on context:
- HTML body:
<→< - HTML attribute: quote-escaped
- URL parameter: percent-encoded
- Inside
<script>: JS-escaped - Inside
<style>: CSS-escaped
{{ . }} placeholder and applies the right escaping. This makes it XSS-safe by default — you'd have to actively bypass it (with template.HTML) to introduce a vulnerability.
- vs Jinja2 (Python): Jinja has full Python-like expressions, control flow, inheritance. Go templates are intentionally much simpler — no while loops, no arbitrary expressions, no Python.
- vs Handlebars/Mustache: Comparable simplicity; Go has a more expressive built-in function set.
- vs JSX/template literals: JSX compiles to code; Go templates are interpreted at runtime.
- vs EJS: EJS embeds full JavaScript. Go forbids that — logic stays in Go code.
- Logic-light by design: the philosophy is to do all real logic in Go and use templates only for presentation.
- HTML pages in standard library web servers.
- Email bodies for transactional email.
- Config file generation from a base template + values (very common in DevOps).
- Code generation via
go generate. - Helm charts in Kubernetes use Go templates for rendering manifests.
- CLI output formatting (e.g.
docker ps --format). - Documentation templates.
html/template is the easiest way to ship safe HTML — never reach for plain string concatenation when rendering user data.
Go's text/template and html/template packages render dynamic text from templates. HTML templates auto-escape to prevent XSS.
import "html/template"
// Parse and execute a template
tmpl := template.Must(
template.New("greeting").Parse("Welcome, {{.name}}! Age: {{.age}}\n"),
)
data := map[string]interface{}{
"name": "John",
"age": 25,
}
tmpl.Execute(os.Stdout, data) // Welcome, John! Age: 25
// template.Must panics if Parse fails — use for known-good templates
// template.New("name").Parse("...") returns (Template, error)
// Multiple named templates
templates := map[string]string{
"welcome": "Welcome, {{.name}}!",
"notification": "{{.name}}, you have: {{.msg}}",
"error": "Oops! Error: {{.err}}",
}
parsed := make(map[string]*template.Template)
for name, tmpl := range templates {
parsed[name] = template.Must(template.New(name).Parse(tmpl))
}
// Use the right template based on context
parsed["welcome"].Execute(os.Stdout, map[string]string{"name": "Alice"})
// Welcome, Alice!
- html/template for web pages — auto-escapes HTML/JS/URLs to prevent XSS by default
- text/template for code/config gen — generating Kubernetes manifests, Terraform files, SQL scripts
- Email templates — Welcome/reset-password emails with user data substitution
- CLI output —
kubectl get pods -o templateuses Go templates for custom output formatting - ParseFiles + ParseGlob — load template directories at startup, render per-request
- Custom funcs — register helpers via
tmpl.Funcs(template.FuncMap{...})for date formatting, math, etc. - Hugo/Helm — both built on Go templates; the most widely-used templating ecosystem
60 Embed Directive
//go:embed, placed immediately above a variable declaration:
import "embed" //go:embed config.json var configBytes []byte //go:embed templates/*.html var tmplFS embed.FS //go:embed README.md var readme stringThe variable can be one of three types:
string— for a single text file (UTF-8).[]byte— for a single file as raw bytes.embed.FS— a virtual filesystem implementingio/fs.FS, for a whole directory tree (with globs).
- Compile-time inclusion: the Go compiler reads the matching files at build time and bakes them into the binary as data sections.
- Read-only: embedded data is immutable at runtime — perfect for static assets.
- No filesystem dependency: the binary works even on machines that don't have the original files.
- Path matching: patterns are interpreted relative to the source file containing the directive.
//go:embed templates/*.htmlincludes all.htmlfiles intemplates/next to the source file. - Recursive: use
//go:embed all:templatesto include hidden files and dotfiles. - Multiple patterns:
//go:embed file1.txt file2.txt dir/*
embed.FS implements the standard io/fs.FS interface, so it composes with any code expecting a filesystem:
// Read a file
data, _ := tmplFS.ReadFile("templates/index.html")
// Walk the tree
fs.WalkDir(tmplFS, ".", func(path string, d fs.DirEntry, err error) error {
// ...
return nil
})
// Use with html/template
tmpl := template.Must(template.ParseFS(tmplFS, "templates/*.html"))
// Use with http.FileServer
http.Handle("/static/", http.FileServer(http.FS(staticFS)))
- vs pre-1.16 Go: you needed third-party tools like
go-bindata,statik, orvfsgenthat generated huge.gofiles containing base64 strings of your assets. Slow build, ugly diffs, frustrating workflow.//go:embedreplaces all of that with a single comment. - vs Java JAR resources: Java has
getResourceAsStream, but resources still live separately in the JAR. Go embeds them into the binary itself as a single artifact. - vs Python
importlib.resources: Python loads from packages at runtime. Go bakes everything in at compile time. - vs Rust
include_str!/include_bytes!: Same idea, but Rust's macros work file-by-file. Go has whole-directory support viaembed.FS.
- HTML templates baked into a web server.
- SQL migrations bundled with the application.
- Default configuration files.
- Static web assets (CSS, JS, images, favicon).
- Language packs / localization data.
- License text and version info.
- Test fixtures in unit tests.
FROM scratch Docker images that have no other files at all.
- Patterns are relative to the source file with the directive —
//go:embedcan't reach above its own directory. - Embedded files cannot change at runtime — for hot reload during development, layer a real-filesystem fallback.
- Binary size grows by the size of the embedded data. Keep it reasonable; for huge assets, ship separately.
- Hidden files require the
all:prefix.
Go 1.16+ lets you embed files directly into your binary at compile time. No external files needed at runtime — perfect for templates, config, static assets.
import (
"embed"
"fmt"
"io/fs"
)
// Embed a single file as string
//go:embed config.txt
var configContent string // file content loaded at compile time
// Embed a single file as bytes
//go:embed logo.png
var logoBytes []byte
// Embed entire directory
//go:embed templates
var templatesFS embed.FS
func main() {
fmt.Println("Config:", configContent)
// Read a file from embedded filesystem
content, _ := templatesFS.ReadFile("templates/index.html")
fmt.Println(string(content))
// Walk embedded directory
fs.WalkDir(templatesFS, "templates", func(path string, d fs.DirEntry, err error) error {
fmt.Println(path)
return nil
})
}
No blank lines between the comment and the variable declaration. The comment is NOT a regular comment — it's a compiler directive. The embedded files are included in the compiled binary, making your app truly self-contained.
- Static web assets — embed CSS/JS/HTML into a single binary; no separate static dir to deploy
- HTML templates — embed your
templates/*.htmldirectory and parse fromembed.FSat startup - Database migrations — embed
migrations/*.sqlso the migration tool ships with the binary - Default config — embed a default
config.yamlthe user can override - SPA backends — embed React/Vue
dist/output and serve viahttp.FileServerFS - Single-binary CLIs — gh, hugo, kubectl all ship one binary because everything is embedded
- Trade-off — binary gets bigger; updating embedded content requires recompile + redeploy
61 HTTP & Networking
net/http standard library package — a complete, production-grade HTTP/1.x and HTTP/2 client and server. Far more than just a primitive: it includes routing (multiplexer), TLS support, request/response abstractions, cookies, file serving, and reverse proxying.The server side centers on the
http.Handler interface:
type Handler interface {
ServeHTTP(w ResponseWriter, r *Request)
}
Anything implementing this interface can serve HTTP. Plus the convenience type http.HandlerFunc that adapts a function to the interface.
- Simplest server:
http.HandleFunc("/path", handler); http.ListenAndServe(":8080", nil) - Mux:
http.NewServeMux()— the standard request router. Go 1.22+ supports method matching and path wildcards:mux.HandleFunc("GET /users/{id}", h). - Custom server:
&http.Server{Addr: ":8080", Handler: mux, ReadTimeout: 5*time.Second, WriteTimeout: 10*time.Second}— always set timeouts in production! - Static files:
http.FileServer(http.Dir("./public")) - Graceful shutdown:
server.Shutdown(ctx)— stops accepting new connections, waits for in-flight requests to finish. - TLS:
http.ListenAndServeTLS(addr, cert, key, handler)
- Quick GET:
resp, err := http.Get("https://api.example.com")— uses the default client (no timeout — DON'T use in production!). - Custom client:
client := &http.Client{Timeout: 10*time.Second} - POST with body:
http.Post(url, "application/json", body) - Build request:
req, _ := http.NewRequestWithContext(ctx, "PUT", url, body); req.Header.Set("Authorization", "Bearer "+token); resp, _ := client.Do(req) - Always close response body:
defer resp.Body.Close()— otherwise connections leak. - Drain body:
io.Copy(io.Discard, resp.Body)before close to enable connection reuse. - Custom transport: set proxies, TLS configs, connection limits via
http.Transport.
func Logger(next http.Handler) http.Handler {
return http.HandlerFunc(func(w http.ResponseWriter, r *http.Request) {
start := time.Now()
next.ServeHTTP(w, r)
log.Printf("%s %s %v", r.Method, r.URL.Path, time.Since(start))
})
}
handler := Logger(Auth(Recovery(myHandler)))
Composition is just function calls. No special framework support needed. This is the foundation of every Go web framework.
- vs Node.js + Express: Express is a thin wrapper over Node's
httpmodule. Go'snet/httpis far more capable on its own — production-ready out of the box. - vs Python + Flask/Django: Python frameworks are essential because the stdlib's
http.serveris a toy. Go's stdlib is the real deal. - vs Java Servlet/Spring: Java requires servlet containers (Tomcat, Jetty) or frameworks. Go's
http.Serveris the container. - Goroutine per request: Go automatically runs every request in its own goroutine. Concurrency is implicit; no thread-pool tuning.
- Used by major projects directly: Caddy, Hugo, Prometheus, etcd, Docker, Kubernetes API server — all use raw
net/http.
net/http first lets you understand any Go web framework — they're all thin wrappers (Gin, Echo, Chi, Fiber). The server scales naturally because each request runs in its own goroutine, and the standard library handles keep-alive, header parsing, multipart forms, content negotiation, gzip, and TLS for you. For most APIs, you don't need a framework at all — net/http + Go 1.22+ routing is enough.- Set timeouts: ReadTimeout, WriteTimeout, IdleTimeout, ReadHeaderTimeout — without them, you're vulnerable to slowloris attacks.
- Limit body size: wrap with
http.MaxBytesReaderin handlers. - Use a custom client with timeout — never use
http.Getdirectly. - Always close response bodies with defer.
- Add recovery middleware to catch panics in handlers.
- Set
http.Server.Handler, not the global default mux. - Run TLS in production with valid certificates.
// Simple server
http.HandleFunc("/hello", func(w http.ResponseWriter, r *http.Request) {
fmt.Fprintf(w, "Hello!") // write response body
})
log.Fatal(http.ListenAndServe(":8080", nil)) // nil = use DefaultServeMux
// Go 1.22+ enhanced routing
mux := http.NewServeMux() // custom router, not the default
mux.HandleFunc("GET /users/{id}", func(w http.ResponseWriter, r *http.Request) {
id := r.PathValue("id") // extract {id} from URL
fmt.Fprintf(w, "User: %s", id)
})
mux.HandleFunc("POST /users", createUser) // method + path matching
// JSON API handler
func getUsers(w http.ResponseWriter, r *http.Request) {
w.Header().Set("Content-Type", "application/json") // set before writing body
json.NewEncoder(w).Encode(users) // stream JSON to client
}
// HTTP Client with timeout
client := &http.Client{Timeout: 10 * time.Second} // always set a timeout
req, _ := http.NewRequestWithContext(ctx, "GET", url, nil) // ctx allows cancellation
req.Header.Set("Authorization", "Bearer token") // set auth header
resp, err := client.Do(req) // send request
defer resp.Body.Close() // always close body to free connection
_ = err // handle error in real code
- net/http is production-ready — many companies (Cloudflare, Uber) run net/http directly without a framework
- ALWAYS set client Timeout — default Client has none; one slow upstream can hang forever
- Reuse http.Client — clients pool TCP connections; creating a new one per call leaks file descriptors
- http.ServeMux Go 1.22+ — supports method matching and path params natively (no need for chi/gin for simple APIs)
- Always defer resp.Body.Close() — even on error; failure to close leaks connections
- http.Server with timeouts — set
ReadTimeout,WriteTimeout,IdleTimeoutto prevent slowloris attacks - Middleware via http.Handler wrapping — auth, logging, recovery, CORS — chain handlers
- http.FileServer — serve static files in one line
62 Testing
testing package plus the go test command. There's no JUnit, no pytest, no Jest to install — testing is part of the language and toolchain. Test files end in _test.go and live next to the code they test, in the same package, so they can access unexported names.The framework supports:
- Unit tests:
func TestXxx(t *testing.T) - Subtests:
t.Run("name", func(t *testing.T) {...})— for table-driven cases. - Parallel tests:
t.Parallel()— runs in parallel with other parallel tests. - Benchmarks:
func BenchmarkXxx(b *testing.B) - Examples:
func ExampleXxx()— appear in godoc AND verify expected output. - Fuzz tests (Go 1.18+):
func FuzzXxx(f *testing.F)— randomized input testing. - Setup/teardown:
TestMainfor package-level,t.Cleanupfor per-test.
t.Error / t.Errorf— mark test as failed but continue.t.Fatal / t.Fatalf— mark failed and stop the test.t.Log / t.Logf— print only on failure (or with-v).t.Skip / t.Skipf— skip the test (with reason).t.Helper()— call inside helper functions so failure messages show the caller's line, not the helper's.t.TempDir()— auto-cleaned temp dir, perfect for file-based tests.t.Cleanup(fn)— register cleanup that runs at test end.t.Setenv(k, v)— set env var for the test, auto-restored.t.Run(name, fn)— create a subtest with its own name.
func TestAdd(t *testing.T) {
cases := []struct {
name string
a, b int
expected int
}{
{"positives", 2, 3, 5},
{"negatives", -2, -3, -5},
{"mixed", -2, 3, 1},
{"zero", 0, 5, 5},
}
for _, tc := range cases {
t.Run(tc.name, func(t *testing.T) {
got := Add(tc.a, tc.b)
if got != tc.expected {
t.Errorf("Add(%d,%d) = %d, want %d", tc.a, tc.b, got, tc.expected)
}
})
}
}
Each case becomes a subtest, runnable individually, with clear failure messages.
- vs JUnit (Java): JUnit needs annotations, build tool plugins, separate test runners. Go has all of that built into
go test. - vs pytest (Python): Pytest has tons of features (fixtures, parametrize, plugins) but requires installation. Go's stdlib covers ~80% with zero deps.
- vs Jest/Mocha (JS): JS test runners are heavy, slow, and config-heavy. Go's
go test ./...just works. - No built-in
assert: you callt.Errorfwith actual/expected values yourself. The community standard for assertions isstretchr/testify, but plain testing is fine for most code. - Tests in same package can access unexported names. Use a
_testpackage suffix (foo_test) for black-box tests. - Fuzz testing built-in — Go 1.18+ added native fuzzing, no separate tool.
go test ./...— run all tests in module.go test -v ./pkg— verbose, show passing tests too.go test -run TestName— run only matching tests (regex).go test -race ./...— with race detector.go test -cover ./...— show coverage.go test -coverprofile=cover.out+go tool cover -html=cover.out— visual coverage report.go test -bench=. -benchmem— run benchmarks with allocation stats.go test -fuzz FuzzName— start fuzzing.go test -count=10— run multiple times to catch flakiness.
Go has a built-in testing framework — no third-party libraries needed. Test files must end with _test.go and test functions must start with Test.
The Function Under Test
// math.go
package math
func Add(a, b int) int {
return a + b
}
Level 1 — Basic Test (Simplest Form)
// math_test.go
package math
import "testing"
func TestAdd(t *testing.T) { // MUST start with Test + uppercase letter
result := Add(2, 3)
expected := 5
if result != expected {
t.Errorf("Add(2, 3) = %d; want %d", result, expected) // marks test as FAILED
}
}
// t.Errorf — logs error but continues running
// t.Fatalf — logs error and STOPS this test immediately
// t.Log — logs info (only shown if test fails or -v flag)
Level 2 — Table-Driven Tests (Idiomatic Go)
Run multiple test cases from a single function. The Go community standard.
func TestAddTableDriven(t *testing.T) {
tests := []struct{ a, b, expected int }{ // slice of anonymous structs
{2, 3, 5},
{0, 0, 0},
{-1, 1, 0},
{100, 200, 300},
}
for _, test := range tests {
result := Add(test.a, test.b)
if result != test.expected {
t.Errorf("Add(%d, %d) = %d; want %d", test.a, test.b, result, test.expected)
}
}
}
Level 3 — Subtests with t.Run (Named Test Cases)
Each case gets its own name — you can run individual subtests and see which specific case failed.
func TestAddSubtests(t *testing.T) {
tests := []struct {
name string
a, b int
expected int
}{
{"positive", 2, 3, 5},
{"zeros", 0, 0, 0},
{"negative", -1, 1, 0},
}
for _, tt := range tests {
t.Run(tt.name, func(t *testing.T) { // creates subtest with name
result := Add(tt.a, tt.b)
if result != tt.expected {
t.Errorf("result = %d; want %d", result, tt.expected)
}
})
}
}
// Run specific subtest: go test -run TestAddSubtests/positive
Level 4 — Real-World Test with Error Handling
func TestDivide(t *testing.T) {
tests := []struct {
name string
a, b float64
want float64
wantErr bool // do we EXPECT an error for this case?
}{
{"normal", 10, 2, 5, false}, // 10/2 = 5, no error
{"zero div", 10, 0, 0, true}, // divide by zero, expect error
{"negatives", -6, 3, -2, false}, // -6/3 = -2, no error
}
for _, tt := range tests {
t.Run(tt.name, func(t *testing.T) {
got, err := Divide(tt.a, tt.b)
if (err != nil) != tt.wantErr { // check error presence matches expectation
t.Errorf("error = %v, wantErr %v", err, tt.wantErr)
}
if got != tt.want {
t.Errorf("got %v, want %v", got, tt.want)
}
})
}
}
Testing Helper Functions
// Validate a slice generation function
func TestGenerateRandomSlice(t *testing.T) {
size := 100
slice := GenerateRandomSlice(size)
if len(slice) != size {
t.Errorf("expected slice size %d, received %d", size, len(slice))
}
// Verify all values are in expected range
for i, v := range slice {
if v < 0 || v >= 100 {
t.Errorf("slice[%d] = %d; want 0-99", i, v)
}
}
}
| Command | Description |
|---|---|
go test | Run all tests in current package |
go test ./... | Run tests in ALL packages recursively |
go test -v | Verbose — show every test name + PASS/FAIL |
go test -run TestAdd | Run only tests matching "TestAdd" |
go test -run TestAdd/positive | Run specific subtest |
go test -count=1 | Disable test caching (force re-run) |
go test -race | Run with race detector enabled |
go test -cover | Show test coverage percentage |
go test -coverprofile=c.out | Generate coverage file |
go tool cover -html=c.out | Open coverage report in browser |
- File must end with
_test.go— build tool ignores test files in production builds - Function must start with
Test+ uppercase letter (e.g.,TestAdd, notTestadd) - Takes exactly one parameter:
*testing.T - Go has no built-in
assert— useif+t.Errorf()(or usetestifypackage for assertions) - Tests are in the same package as the code they test — they can access unexported functions
- Table-driven tests — Go community standard; one function tests dozens of cases with clear failure messages
- httptest.NewRecorder + httptest.NewRequest — testing HTTP handlers without a real server
- httptest.NewServer — fake an upstream API to test your client code
- testify (assert/require) — most popular assertion library;
require.NoError(t, err)for clearer tests - t.TempDir() and t.Cleanup() — auto-cleanup test artifacts
- Integration tests in
_integration_test.go— gated by// +build integrationtag, run nightly with-tags=integration - Mock interfaces, not concrete types — define an interface in the consumer package, mock it in tests
- go test -race in CI — catches data races automatically
63 Benchmarking
testing package — write a function with the signature func BenchmarkXxx(b *testing.B) in any _test.go file, then run with go test -bench=. -benchmem. The framework auto-tunes b.N (the loop count) until measurements are statistically stable, then reports time-per-op and (with -benchmem) bytes and allocations per op.Sample output:
BenchmarkAdd-12 1000000000 0.32 ns/op 0 B/op 0 allocs/op BenchmarkParse-12 2548437 482 ns/op 240 B/op 3 allocs/op // ↑ CPUs ↑ iters ↑ ns/op ↑ memory ↑ allocations
func BenchmarkParse(b *testing.B) {
data := loadFixture() // setup BEFORE loop
b.ResetTimer() // exclude setup from measurement
for i := 0; i < b.N; i++ {
_ = Parse(data)
}
}
Key methods on *testing.B:
b.N— the loop count, set by the framework (could be 100, could be billions).b.ResetTimer()— exclude prior work from measurement.b.StopTimer()/b.StartTimer()— pause/resume mid-benchmark for per-iteration setup.b.ReportAllocs()— force allocation reporting (auto with-benchmem).b.RunParallel(fn)— run the body in many goroutines for concurrent benchmarks.b.Run(name, fn)— sub-benchmarks for table-driven cases.
benchstat:
$ go test -bench=. -count=10 > old.txt # (make changes) $ go test -bench=. -count=10 > new.txt $ benchstat old.txt new.txt name old time/op new time/op delta Parse-12 480ns ± 3% 280ns ± 2% -41.67%
benchstat performs a statistical test and rejects noise — only reports differences that are real.
- vs Java JMH: JMH is a separate library/tool with annotations. Go's benchmarks are part of
go test. - vs Linux
perf:perfprofiles the whole process at the system level. Go benchmarks measure specific functions in isolation, with auto-tuned loop counts. - vs ad-hoc
time.Now()measurements: Auto-tuningb.Nhandles warm-up, JIT (none in Go but cache effects), and statistical noise much better than hand-rolled timing. - vs Python timeit / pytest-benchmark: Comparable purpose; Go's is built into the test runner, not a separate command.
- vs Rust
cargo bench: Similar in spirit. Rust's stable bench is via the Criterion crate; Go has it in stdlib.
-benchmem + pprof is how Go programmers find allocation hotspots and shave off latency in critical paths like serialization, parsing, hashing, and concurrent state machines. Without benchmarks, performance optimization is guesswork — and most guesses are wrong.- Always benchmark with
-benchmem— allocations dominate Go performance more than raw CPU usually. - Run multiple times with
-count=10and usebenchstatto compare. - Use realistic input sizes — micro-cases can mislead.
- Use
b.ResetTimerafter expensive setup. - Beware compiler optimizations eliminating "dead" code — assign results to a sink variable.
- Lock GOMAXPROCS in CI or use a quiet machine to reduce noise.
- Profile before optimizing — benchmarking the wrong function is worse than not benchmarking at all.
Go has built-in benchmarking in the testing package. Benchmark functions start with Benchmark and take *testing.B. The framework automatically determines b.N — how many iterations to run for stable results.
Basic Benchmark
// File: math_test.go
func BenchmarkAdd(b *testing.B) { // must start with Benchmark
for range b.N { // b.N is auto-tuned by framework (could be millions)
Add(2, 3)
}
}
// Output: BenchmarkAdd-12 1000000000 0.3180 ns/op
// name-CPUs iterations time per operation
Comparing Input Sizes
Run the same function with different input sizes to understand how performance scales.
func BenchmarkAddSmallInput(b *testing.B) {
for range b.N { Add(2, 3) }
}
func BenchmarkAddMediumInput(b *testing.B) {
for range b.N { Add(200, 300) }
}
func BenchmarkAddLargeInput(b *testing.B) {
for range b.N { Add(2000, 3000) }
}
// For Add, all three will be similar — integer addition is O(1)
// But for algorithms like sorting, you'd see clear differences
Benchmark with Setup — b.ResetTimer()
Exclude setup time (slice creation, file loading, etc.) from the measurement.
func GenerateRandomSlice(size int) []int {
slice := make([]int, size)
for i := range slice { slice[i] = rand.Intn(100) }
return slice
}
func SumSlice(slice []int) int {
sum := 0
for _, v := range slice { sum += v }
return sum
}
// Benchmark the generation itself
func BenchmarkGenerateRandomSlice(b *testing.B) {
for range b.N {
GenerateRandomSlice(1000)
}
}
// Benchmark ONLY SumSlice — exclude slice creation
func BenchmarkSumSlice(b *testing.B) {
slice := GenerateRandomSlice(1000) // setup — NOT measured
b.ResetTimer() // reset timer AFTER setup
for range b.N {
SumSlice(slice) // only this is measured
}
}
b.ResetTimer()— reset once after one-time setup (most common)b.StopTimer()/b.StartTimer()— pause/resume timer inside the loop for per-iteration setup that shouldn't be measured
| Command | Description |
|---|---|
go test -bench=. | Run all benchmarks in current package |
go test -bench=Sum | Run benchmarks matching "Sum" |
go test -bench=. -benchmem | Include memory allocation stats (allocs/op, B/op) |
go test -bench=. -count=5 | Run each benchmark 5 times (for stable averages) |
go test -bench=. -benchtime=5s | Run for 5 seconds minimum per benchmark |
go test -bench=. -cpuprofile=cpu.out | Generate CPU profile for go tool pprof |
- Compare alternatives — benchmark
strings.Buildervs+=concat to prove the difference - Detect regressions — run benchmarks in CI; tools like
benchstatcompare results between commits - Profile-driven optimization — combine
-cpuprofile+go tool pprofto find hot spots before optimizing blindly - Allocation hunting —
-benchmemshows B/op and allocs/op; reducing allocations is often the biggest perf win - Always reset timer after setup —
b.ResetTimer()prevents one-time setup from skewing results - b.RunParallel — for testing concurrent code under load (e.g., a thread-safe cache)
- Avoid micro-optimization — benchmark the right level (a service handler, not a single line)
64 OS Processes & Exec
os/exec package — Go's facility for spawning external commands from your program. The workflow:
- Build:
cmd := exec.Command(name, args...)or modernexec.CommandContext(ctx, name, args...). - Configure: set
cmd.Stdin,cmd.Stdout,cmd.Stderr,cmd.Dir,cmd.Env. - Run: pick one of the execution methods.
cmd.Run()— start and wait for completion. Returns nil on exit code 0.cmd.Start()— start without waiting. Pair withcmd.Wait().cmd.Output()— capture stdout, return as[]byte.cmd.CombinedOutput()— capture stdout AND stderr together.cmd.StdinPipe()/StdoutPipe()/StderrPipe()— get io.Reader/Writer for streaming.cmd.Process.Kill()— force-terminate the child.cmd.ProcessState— exit code, success, etc., available after Wait/Run.
- Working directory:
cmd.Dir = "/tmp" - Environment:
cmd.Env = append(os.Environ(), "KEY=value") - Inherit terminal:
cmd.Stdin = os.Stdin; cmd.Stdout = os.Stdout; cmd.Stderr = os.Stderr— useful for interactive tools. - Pipe input:
cmd.Stdin = strings.NewReader("input data") - Capture to buffer:
var buf bytes.Buffer; cmd.Stdout = &buf - Tee output:
cmd.Stdout = io.MultiWriter(os.Stdout, &buf)— to terminal AND a buffer. - Process groups: set
cmd.SysProcAttrfor platform-specific options.
- vs C
system()or PHPshell_exec(): Those pass the command through a shell, which means shell injection if any arg contains user input.exec.Commandtakes argv as separate strings — no shell, no injection by default. - vs Python
subprocess: Same shape, similar API. Python's default isshell=Falsetoo, but it's easy to forget. Go forces the safe form. - vs Java
ProcessBuilder: Comparable scope; Go is more concise. - vs Node.js
child_process: Node hasspawn(safe) andexec(shell-based). Same trap. - If you NEED a shell:
exec.Command("sh", "-c", "ls -l | wc -l")— but be careful with user input. - Context cancellation:
exec.CommandContext(ctx, ...)kills the child process when the context is cancelled. Critical for long-running operations.
- Wrapping CLI tools:
git,ffmpeg,kubectl,docker,imagemagick,aws. - Running build steps in custom build systems.
- System administration tools that compose existing utilities.
- Sandboxing untrusted code in a separate process.
- Calling scripts in other languages (Python, shell, Node).
exec.CommandContext for anything user-triggered so a hung child can be cancelled cleanly.
- Never concatenate user input into a shell command. Pass as separate args.
- Validate the binary path —
exec.LookPathfinds it on PATH. - Set timeouts via context — orphaned child processes are hard to clean up.
- Capture stderr for debugging — silent failures are common otherwise.
- Beware exit code 0 from a wrapper script that swallowed errors — verify outputs too.
- Process groups: on Linux, child processes can outlive the parent unless you set up a process group and signal it.
The os/exec package runs external commands from Go. You can capture output, pipe input, and manage process lifecycle.
import "os/exec"
// Run a command and capture output
cmd := exec.Command("ls", "-l")
output, err := cmd.CombinedOutput() // stdout + stderr combined
if err != nil { fmt.Println("Error:", err); return }
fmt.Println(string(output))
// Pipe input to a command
cmd = exec.Command("grep", "foo")
cmd.Stdin = strings.NewReader("food is good\nbar\nbaz\n")
out, _ := cmd.Output() // "food is good\n"
// Read environment variable from command
cmd = exec.Command("printenv", "SHELL")
out, _ = cmd.Output()
fmt.Println(string(out)) // e.g., "/bin/zsh"
// Start a long-running process and kill it
cmd = exec.Command("sleep", "60")
cmd.Start() // non-blocking start
time.Sleep(2 * time.Second)
cmd.Process.Kill() // kill after 2 seconds
cmd.Wait() // wait for process to exit
- Image/video processing — invoking
ffmpeg,imagemagickfrom Go services - Git automation — running
git clone,git pull,git rev-parsefor build systems and CI tools - System scripts — wrapping shell scripts/binaries when there's no native Go library
- Use exec.CommandContext for timeouts — kills the process when context expires; prevents zombie processes
- NEVER pass user input to
sh -c— command injection vulnerability; pass args as separate strings - Capture stdout AND stderr — many tools log errors to stderr; use
CombinedOutputor wire both pipes - Stream large output — for long-running commands, use
cmd.StdoutPipe+ bufio.Scanner
65 Command Line Args & Flags
os.Args— a raw[]stringof all command-line arguments.os.Args[0]is the program path;os.Args[1:]are the actual args.flagpackage — a typed parser that handles options like-port=8080or--name=foo. Declare flags withflag.String/Int/Bool/Duration/Float64, then callflag.Parse()inmain().
var (
port = flag.Int("port", 8080, "server port")
name = flag.String("name", "world", "who to greet")
verbose = flag.Bool("v", false, "verbose output")
)
func main() {
flag.Parse() // must be called before reading flag values
fmt.Printf("Hello %s on port %d\n", *name, *port)
// positional args after flags:
for _, arg := range flag.Args() { fmt.Println(arg) }
}
Or use the Var form to bind to existing variables: flag.IntVar(&port, "port", 8080, "...").
- Built-in types:
String,Int,Int64,Uint,Bool,Float64,Duration(time.Duration). - Var forms:
StringVar,IntVar, etc., bind to existing variables. - Custom flags: implement the
flag.Valueinterface (String() string+Set(string) error) for any type. flag.Args()— non-flag positional arguments after parsing.flag.NArg()— count of positional arguments.flag.Usage— override the default--helpoutput.- Custom FlagSet:
flag.NewFlagSetfor building separate flag groups (per subcommand).
flag package accepts several forms — both single and double dash work:
-name value-name=value--name value--name=value- Boolean:
-v,--v,-v=true,-v=false - NOT supported: grouped short flags like
-vrf(each must be separate).
- vs Python
argparse: argparse has many features (subcommands, mutually exclusive groups, etc.) but more boilerplate. Go'sflagis simpler. - vs Cobra (community standard): for full CLI tools with subcommands, hierarchical help, shell completions — use
spf13/cobra. It's howkubectl,hugo,helm,gh, and most Go CLIs are built. - vs Java commons-cli / picocli: Comparable feature set in those Java libs.
- vs Node.js commander/yargs: Similar in spirit; Cobra is the equivalent for Go.
- Single-dash long flags (
-name) are the Go convention but double-dash works too.
flag package is enough and brings zero dependencies. Knowing it lets you read the source of every standard Go binary (go, gofmt, golangci-lint) without first learning a framework. For larger CLIs with subcommands and rich help, graduate to Cobra.Raw Arguments (os.Args)
import "os"
// os.Args is a slice of all command-line arguments
fmt.Println("Command:", os.Args[0]) // program name
for i, arg := range os.Args {
fmt.Println("Arg", i, ":", arg)
}
// go run main.go hello world
// Arg 0: /tmp/go-build.../main
// Arg 1: hello
// Arg 2: world
Flags (flag package)
import "flag"
// Define flags with default values
var name string
var age int
var male bool
flag.StringVar(&name, "name", "John", "Name of the user")
flag.IntVar(&age, "age", 18, "Age of the user")
flag.BoolVar(&male, "male", true, "Gender of the user")
flag.Parse() // parse the command-line flags
fmt.Println("Name:", name, "Age:", age, "Male:", male)
// go run main.go -name=Alice -age=30 -male=false
Subcommands
// Create separate flag sets for subcommands
sub1 := flag.NewFlagSet("deploy", flag.ExitOnError)
sub2 := flag.NewFlagSet("status", flag.ExitOnError)
deployEnv := sub1.String("env", "dev", "Target environment")
statusVerbose := sub2.Bool("verbose", false, "Verbose output")
if len(os.Args) < 2 { fmt.Println("Expected subcommand"); os.Exit(1) }
switch os.Args[1] {
case "deploy":
sub1.Parse(os.Args[2:])
fmt.Println("Deploying to:", *deployEnv)
case "status":
sub2.Parse(os.Args[2:])
fmt.Println("Verbose:", *statusVerbose)
}
// go run main.go deploy -env=prod
// go run main.go status -verbose
- stdlib flag for simple tools — single-binary CLIs with a handful of options
- spf13/cobra for complex CLIs — kubectl, helm, hugo all use cobra; supports nested subcommands, autocompletion, help generation
- urfave/cli — alternative to cobra with a simpler API
- Validate flags after Parse — required flags, mutually exclusive options aren't enforced by stdlib
- Use env vars as flag defaults —
flag.StringVar(&url, "url", os.Getenv("API_URL"), "...")— supports both - os.Args[0] for tool name — used when symlinked binaries change behavior (busybox-style)
66 Environment Variables
os package provides:
os.Getenv("KEY")— returns the value, or""if not set. No way to distinguish "set to empty" from "not set".os.LookupEnv("KEY")— returns(value, ok bool);okis true if set (even if empty).os.Setenv("KEY", "value")— set for this process and any children spawned afterward.os.Unsetenv("KEY")— remove a variable.os.Environ()— returns all environment as[]stringinKEY=valueform.os.ExpandEnv("$HOME/foo")— substitute env var references.
- vs other languages: Same OS-level concept everywhere. Go's API is minimalist — just six functions.
- No
.envfile support in stdlib. Most Go projects usegodotenvin dev to load.envfiles into the process env, and rely on the deployment platform in production. - vs config files (YAML/TOML/JSON): env vars are great for secrets and per-environment differences but bad for structured data — use a config file for nested structures.
- vs CLI flags: env vars persist across invocations and are easier to set in containers. Flags are easier for ad-hoc one-off changes.
- Configuration is what differs between deploys (dev / staging / prod).
- Code shouldn't change between deploys.
- Therefore: configuration belongs outside the code.
- Env vars are universal and language-agnostic — every platform supports them.
- They never accidentally get checked into git (unlike
config.json).
- Kubernetes: Pod spec
env:field, ConfigMaps, Secrets. - Docker:
-e KEY=valueondocker runorenvironment:in compose files. - systemd:
Environment=in unit files. - AWS ECS: task definition
environmentandsecretsblocks. - Heroku/Render/Fly.io: dashboard or CLI commands.
- CI/CD systems: GitHub Actions secrets, GitLab CI variables.
- Local development:
.envfile loaded bygodotenv, ordirenvshell extension.
- Database URLs and connection strings.
- API keys and secrets (Stripe, AWS, OpenAI).
- Log levels (DEBUG / INFO / WARN).
- Port numbers and bind addresses.
- Feature flags for A/B testing or kill switches.
- Service discovery URLs in microservices.
- Anything that varies between dev/stage/prod and shouldn't be in the binary.
- Don't store secrets in
.envfiles committed to git — use a secrets manager (Vault, AWS Secrets Manager, GCP Secret Manager). - Validate env vars at startup — fail fast if a required one is missing.
- Use a typed config struct populated from env: libraries like
kelseyhightower/envconfig,caarlos0/env, orspf13/viper. - Document required vars in a
.env.examplecommitted to the repo. - Don't print env vars in logs — they may contain secrets.
import "os"
// Read environment variables
user := os.Getenv("USER") // returns "" if not set
home := os.Getenv("HOME") // e.g., "/home/john"
// Set an environment variable (for this process only)
os.Setenv("FRUIT", "APPLE")
fmt.Println(os.Getenv("FRUIT")) // "APPLE"
// Unset
os.Unsetenv("FRUIT")
fmt.Println(os.Getenv("FRUIT")) // "" (empty)
// Check if a var exists (Getenv can't distinguish "" from unset)
val, exists := os.LookupEnv("DATABASE_URL")
if !exists { fmt.Println("DATABASE_URL not set!") }
// List ALL environment variables
for _, e := range os.Environ() {
pair := strings.SplitN(e, "=", 2) // split key=value
fmt.Printf("%s = %s\n", pair[0], pair[1])
}
- Configuration over code — DATABASE_URL, REDIS_URL, API keys live in env vars, NEVER in committed config files
- kelseyhightower/envconfig — populate a Config struct from env vars with tags:
envconfig:"PORT" default:"8080" - spf13/viper — multi-source config (env + YAML + flags) with precedence
- Use LookupEnv to detect missing —
Getenvreturns empty string for both unset and empty - Don't log env vars — they often contain secrets; allowlist what's safe to log
- godotenv for local dev — loads
.envfile into env (don't use in production; CI/runtime should set vars directly) - Kubernetes Secrets / ConfigMaps — both expose data as env vars in pods
67 Logging
log— the original, line-based logger. Prefix + flags + free-form text. No levels. Fine for tiny tools, painful for production.log/slog(Go 1.21+) — Go's modern structured logging package. Supports leveled logging (Debug/Info/Warn/Error), key-value attributes, context propagation, and pluggable handlers for text or JSON output.
slog:
- Zap (Uber) — extremely fast, zero-allocation in hot paths.
- Zerolog — chained API, also very fast.
- Logrus — older, more popular than the others historically, slower.
// Default text handler
slog.Info("user login", "user_id", 42, "ip", "1.2.3.4")
// → time=2026-04-08T10:00:00 level=INFO msg="user login" user_id=42 ip=1.2.3.4
// JSON handler for production
h := slog.NewJSONHandler(os.Stdout, &slog.HandlerOptions{Level: slog.LevelInfo})
logger := slog.New(h)
logger.Error("db query failed", "err", err, "query", q)
// → {"time":"...","level":"ERROR","msg":"db query failed","err":"...","query":"..."}
// Add context that propagates
logger = logger.With("request_id", reqID, "user", userID)
logger.Info("processing") // every log includes request_id and user
slog defines four standard levels with numeric values that allow custom in-betweens:
slog.LevelDebug(-4) — verbose internal state.slog.LevelInfo(0) — normal events.slog.LevelWarn(4) — something concerning but not failing.slog.LevelError(8) — actual failures.- Custom levels can use any integer (e.g. Trace = -8, Fatal = 12).
- Set the cutoff with
HandlerOptions.Level.
log):
log.Printf("user %d login from %s in %v", id, ip, dur)
// "user 42 login from 1.2.3.4 in 350ms"
Structured (slog with JSON handler):
slog.Info("user login", "user_id", id, "ip", ip, "duration_ms", dur.Milliseconds())
// {"level":"INFO","msg":"user login","user_id":42,"ip":"1.2.3.4","duration_ms":350}
The structured form is queryable: you can filter by user_id=42, alert on duration_ms>1000, group by ip. The unstructured form requires regex parsing.
- vs Java SLF4J / Logback: Comparable structured logging support; SLF4J has a longer history and richer ecosystem. Go's slog is simpler but newer.
- vs Python
logging: Python's stdlib logging is older and uses %-formatting; structured support comes via third-party libraries like structlog. - vs Node.js
winston/pino: Pino is JSON-first like slog. Winston supports many transports. - vs Rust
log+tracing: Rust's tracing crate is similar in spirit to slog, with structured contextual logging. - slog as a façade: slog is intentionally designed so other loggers (Zap, Zerolog) can be plugged in as handlers, unifying the ecosystem.
slog + the JSON handler. For legacy code, the older log package is still fine for simple needs but limits future observability.Standard Library (log package)
import "log"
// Basic logging
log.Println("This is a log message.") // 2024/07/30 12:30:45 This is a log message.
// Customize prefix and flags
log.SetPrefix("INFO: ")
log.SetFlags(log.Ldate | log.Ltime | log.Lshortfile)
log.Println("With file info") // INFO: 2024/07/30 12:30:45 main.go:15: With file info
// Multiple loggers with different levels
infoLog := log.New(os.Stdout, "INFO: ", log.Ldate|log.Ltime|log.Lshortfile)
warnLog := log.New(os.Stdout, "WARN: ", log.Ldate|log.Ltime)
errorLog := log.New(os.Stderr, "ERROR: ", log.Ldate|log.Ltime|log.Lshortfile)
infoLog.Println("Server started")
warnLog.Println("Cache miss")
errorLog.Println("Database connection failed")
// Log to a file
file, _ := os.OpenFile("app.log", os.O_CREATE|os.O_WRONLY|os.O_APPEND, 0666)
defer file.Close()
fileLogger := log.New(file, "APP: ", log.Ldate|log.Ltime)
fileLogger.Println("Logged to file")
// Fatal and Panic
log.Fatal("Fatal error") // logs + os.Exit(1)
log.Panic("Panic!") // logs + panic()
Uber's Zap (structured logging)
// go get go.uber.org/zap
import "go.uber.org/zap"
logger, _ := zap.NewProduction() // structured JSON output
defer logger.Sync() // flush buffered logs
logger.Info("User logged in",
zap.String("username", "John"),
zap.String("method", "GET"),
)
// {"level":"info","ts":1722345600,"msg":"User logged in","username":"John","method":"GET"}
- log/slog (Go 1.21+) — stdlib structured logging; replaces zap/zerolog for new projects in many cases
- Structured / JSON output — required for Elasticsearch, Datadog, Loki, CloudWatch — no parsing of free-form text
- Log levels — DEBUG (off in prod), INFO (normal events), WARN (something off), ERROR (failures), FATAL (crash)
- Context-aware logging — extract request ID, user ID from
context.Contextand include on every log line - Don't log secrets — passwords, API keys, PII; allowlist what's safe and redact the rest
- Sampling for high-volume — log 1 in 1000 trace events to control cost (zap supports this natively)
- Async logging — buffer + flush; never block business logic on log writes
- zerolog vs zap — zerolog is slightly faster + smaller API; both blow stdlib log out of the water for high-throughput
Logrus (structured + levels)
// go get github.com/sirupsen/logrus
import "github.com/sirupsen/logrus"
log := logrus.New()
log.SetLevel(logrus.InfoLevel)
log.SetFormatter(&logrus.JSONFormatter{}) // JSON output
log.Info("This is an info message.")
log.Warn("This is a warning.")
log.Error("This is an error.")
// Structured logging with fields
log.WithFields(logrus.Fields{
"username": "John",
"method": "GET",
}).Info("User logged in.")
68 Reflection
reflect package — Go's facility for inspecting and manipulating types and values at runtime. The two main entry points:
reflect.TypeOf(v)— returns areflect.Type, the type metadata ofv.reflect.ValueOf(v)— returns areflect.Value, the actual value wrapped for runtime introspection.
encoding/json, fmt, ORMs, validators, dependency injection containers, and any library that needs to operate on user-defined types it doesn't know in advance.
reflect.Type— the static type of a value (e.g.,main.User,*int,[]string). Use.Name(),.PkgPath(),.NumField(),.Field(i),.Method(i), etc.reflect.Value— the actual value at runtime. Use.Interface()to convert back toany,.Field(i),.Method(i),.Set(v)(only for addressable values),.Call(args), etc.reflect.Kind— the underlying primitive category:Int,String,Struct,Slice,Map,Ptr, etc. A customtype MyInt inthas Type=MyInt but Kind=int.
- Iterate struct fields:
t := reflect.TypeOf(x); for i := 0; i < t.NumField(); i++ { f := t.Field(i); ... } - Read struct tag:
t.Field(i).Tag.Get("json") - Get field value:
v := reflect.ValueOf(x); v.Field(i).Interface() - Set field value: requires the value to be addressable (pass
&x):v := reflect.ValueOf(&x).Elem(); v.Field(i).SetString("new") - Call a method:
v.MethodByName("Save").Call(nil) - Check interface satisfaction:
t.Implements(reflect.TypeOf((*io.Reader)(nil)).Elem()) - Create a new value:
reflect.New(t)— returns areflect.Valueof type*T. - Deep equality:
reflect.DeepEqual(a, b)— works on any types, including maps and slices.
- vs Java reflection: Java reflection is more ergonomic — every class has
.getDeclaredFields(),.getMethods(), etc. Go's API is more verbose but more powerful for type construction. - vs Python: Python is "everything is reflection" — you can introspect anything trivially. Go intentionally makes reflection feel awkward to discourage casual use.
- vs C++ template metaprogramming: Compile-time reflection. Go's is runtime-only.
- vs Rust macros: Rust uses procedural macros for similar use cases (serde derive). Go uses runtime reflection plus optional code generation.
- Slow: reflect-based code is typically 10–100× slower than direct typed access. Avoid in hot paths.
- JSON/XML/YAML marshalers — generic serialization.
- ORMs / query builders — map structs to database rows.
- Validators — read tags, apply rules.
- Dependency injection containers — wire types together.
- Test helpers like
reflect.DeepEqualfor comparisons. - Generic printers and inspectors like
fmt.Printf("%+v", x).
- Slow: use reflection only when generics/interfaces won't work.
- Lose type safety: errors that would be caught at compile time become runtime panics.
- Setting unexported fields requires
unsafeor panics. - Setting through a non-addressable value (e.g.
reflect.ValueOf(x).Field(0).Set(...)) panics — must pass a pointer and use.Elem(). - Complex APIs: it's easy to write reflection code that misbehaves on edge cases.
The reflect package lets you inspect and manipulate types and values at runtime. It powers encoding/json, ORMs, and any library that needs to work with arbitrary types. Two core functions: reflect.TypeOf() returns the type, reflect.ValueOf() returns the value.
Core Concepts — Type, Value, Kind
import "reflect"
x := 42
v := reflect.ValueOf(x) // runtime value representation
t := v.Type() // runtime type representation
fmt.Println("Value:", v) // 42
fmt.Println("Type:", t) // int
fmt.Println("Kind:", t.Kind()) // int (the underlying primitive kind)
fmt.Println("Is Zero:", v.IsZero()) // false
// Type = Go type name (e.g., MyInt, Person)
// Kind = underlying primitive (e.g., int, struct, slice, ptr)
// A custom `type MyInt int` has Type=MyInt but Kind=int
// Interface type inspection
var itf interface{} = "Hello"
v3 := reflect.ValueOf(itf)
if v3.Kind() == reflect.String {
fmt.Println("String value:", v3.String()) // "Hello"
}
Type is the declared Go type (the name you wrote in your source code). Kind is the underlying category Go uses internally — one of a fixed set of ~26 values like Int, String, Struct, Slice, Ptr, Map, Chan, Func, Interface.
For a built-in like int, both look identical — that's why the example above prints int twice. The difference shows up the moment you define a custom type:
type UserID int // custom type built on int
type Person struct { Name string }
id := UserID(42)
p := Person{Name: "Alice"}
t1 := reflect.TypeOf(id)
fmt.Println(t1) // main.UserID ← the declared name
fmt.Println(t1.Kind()) // int ← the underlying primitive
t2 := reflect.TypeOf(p)
fmt.Println(t2) // main.Person ← the declared name
fmt.Println(t2.Kind()) // struct ← the category
t3 := reflect.TypeOf(&p)
fmt.Println(t3) // *main.Person ← pointer to Person
fmt.Println(t3.Kind()) // ptr ← it's a pointer
When to use which:
- Use
Typewhen you care about identity — "is this exactly atime.Time?", reading struct field names, comparing two types for equality, reading struct tags. - Use
Kindwhen you care about shape — "is this any kind of integer?", "is this a struct so I can iterate fields?", "is this a pointer so I need to call.Elem()?".Kindis what you switch on.
// Typical reflect dispatch — branch on Kind, not Type
switch v.Kind() {
case reflect.Int, reflect.Int8, reflect.Int16, reflect.Int32, reflect.Int64:
fmt.Println("some integer:", v.Int())
case reflect.String:
fmt.Println("string:", v.String())
case reflect.Struct:
for i := 0; i < v.NumField(); i++ { /* walk fields */ }
case reflect.Ptr:
v = v.Elem() // unwrap and recurse
}
If you switched on Type instead, you'd have to list every possible named type in the universe — including UserID, OrderID, etc. Kind collapses all of those into the single bucket reflect.Int, which is exactly what a JSON encoder or ORM needs.
Go is statically typed: normally the compiler knows every type at compile time and you write code against specific types. But sometimes you need to write a function that works on any type — and you don't know what that type is until the program is running. That's the gap reflect fills.
Concrete examples where you literally cannot avoid reflection:
json.Marshal(anyValue)— the standard library doesn't know about your structs. At runtime it walks the value with reflect, reads each field's name andjson:"..."tag, and emits the right output. Same foryaml,xml,protobuf,bson.- ORMs & database libraries (GORM, sqlx, ent) — given a
Userstruct, they need to figure out column names, scan rows into fields, and build SQL — all without you hand-writing mapping code for every model. - Validation libraries (
go-playground/validator) — read tags likevalidate:"required,email"off arbitrary structs. - Dependency injection / wire frameworks — inspect a constructor's parameter types and supply matching instances from a registry.
- Test helpers like
reflect.DeepEqual— compare two values of unknown type field by field. - Generic printers / debuggers —
fmt.Printf("%+v", x)uses reflect under the hood to walk into structs and print field names.
The key insight: without reflect, every library author would have to ask you to write a ToJSON() method on every struct, a Scan() method for every database row, a Validate() method for every form, and so on. Reflect is the escape hatch that lets one library work with types it has never seen.
When NOT to reach for reflect: in your own application code. It's 10–100× slower than direct calls, errors surface only at runtime, and the code is hard to read. Since Go 1.18, generics cover most "works for any type" needs in a type-safe, compile-time way — use generics first, reach for reflect only when you genuinely need to inspect unknown types (e.g., reading user-defined struct tags).
Modifying Values via Reflection
Must pass a pointer — reflect.ValueOf(x) gets a copy. Use reflect.ValueOf(&x).Elem() to get the addressable original.
y := 10
v := reflect.ValueOf(&y).Elem() // &y = pointer, .Elem() = dereference
fmt.Println("Before:", v.Int()) // 10
v.SetInt(18) // modify the original y
fmt.Println("After:", y) // 18 — y is changed!
// Without pointer: reflect.ValueOf(y) is unaddressable — SetInt would panic
Working with Structs & Fields
type Person struct {
Name string // exported — CanSet() = true
age int // unexported — CanSet() = false
}
p := Person{Name: "Alice", age: 30}
v := reflect.ValueOf(p)
// Read all fields (works for both exported and unexported)
for i := 0; i < v.NumField(); i++ {
fmt.Printf("Field %d: %v\n", i, v.Field(i))
}
// Field 0: Alice
// Field 1: 30
// Modify exported fields (must pass pointer)
v1 := reflect.ValueOf(&p).Elem()
nameField := v1.FieldByName("Name")
if nameField.CanSet() {
nameField.SetString("Jane") // OK — Name is exported
}
fmt.Println(p) // {Jane 30}
// ageField := v1.FieldByName("age")
// ageField.CanSet() → false — unexported fields CANNOT be modified
// Read struct tags via TypeOf
t := reflect.TypeOf(p)
for i := 0; i < t.NumField(); i++ {
f := t.Field(i)
fmt.Println(f.Name, f.Type, f.Tag.Get("json"))
}
Working with Methods — Dynamic Invocation
type Greeter struct{}
func (g Greeter) Greet(fname, lname string) string {
return "Hello " + fname + " " + lname
}
g := Greeter{}
t := reflect.TypeOf(g)
v := reflect.ValueOf(g)
// List all methods
for i := 0; i < t.NumMethod(); i++ {
method := t.Method(i)
fmt.Printf("Method %d: %s\n", i, method.Name) // "Greet"
}
// Call method by name — dynamically!
m := v.MethodByName("Greet")
results := m.Call([]reflect.Value{
reflect.ValueOf("Alice"), // first arg
reflect.ValueOf("Doe"), // second arg
})
fmt.Println(results[0].String()) // "Hello Alice Doe"
// Call() takes []reflect.Value and returns []reflect.Value
// You must wrap every argument with reflect.ValueOf()
[]reflect.Value{reflect.ValueOf(...)} syntax?When you call a function normally, the compiler knows the function name, how many arguments it takes, and the type of each. With reflection, you're calling a function dynamically at runtime — the compiler knows none of that — so you have to manually package everything in a way the reflection system understands.
1. reflect.ValueOf("Alice") — wrapping each argument
Reflection can't deal with raw Go values directly; it needs a generic "box" (reflect.Value) that can hold any type uniformly — string, int, struct, slice, etc. ValueOf creates that box:
"Alice" → [reflect.Value: type=string, value="Alice"]
2. []reflect.Value{...} — the slice of args
The signature of Call is:
func (v Value) Call(in []reflect.Value) []reflect.Value
It takes a slice because a function can have any number of arguments (0, 1, 2, 10...) and reflection doesn't know how many at compile time. A slice is the only way to pass a variable-length list of already-wrapped values.
Normal vs reflective call
// Normal — compiler does all the work
greet("Alice", "Doe")
// Reflective — you do all the work manually
fn := reflect.ValueOf(greet)
args := []reflect.Value{
reflect.ValueOf("Alice"),
reflect.ValueOf("Doe"),
}
fn.Call(args)
Mental model: Reflection is like sending a package through the mail. You can't hand someone a loose item — you have to box it (reflect.ValueOf) and put all your boxes in a shipping container ([]reflect.Value) before the postal system (Call) will accept it. The verbosity isn't Go being annoying — it's the cost of giving up compile-time type safety in exchange for runtime flexibility. You only pay this tax when you don't know the function ahead of time (web frameworks routing to handlers, ORMs setting struct fields, test runners, JSON decoders).
- Use: JSON/XML marshallers, ORM field mapping, dependency injection, struct tag reading
- Avoid: Normal application code — it's slow (10-100x), not type-safe, and errors happen at runtime instead of compile time
- Prefer generics (Go 1.18+) for type-safe generic code — they're faster and catch errors at compile time
CanSet()returns false for unexported fields and non-pointer values — always check before callingSet*()
- encoding/json — uses reflect to read struct tags and field types at runtime
- ORMs (GORM, sqlx, ent) — map struct fields to DB columns via reflection
- go-playground/validator — reads
validate:"..."tags off arbitrary structs - Wire/dependency injection — inspects constructor signatures to wire up dependencies
- Test helpers —
reflect.DeepEqualfor comparing values of unknown type in assertions - fmt %v/%+v — uses reflect to walk into structs and slices for debug printing
- YOU should rarely write reflect code — stick to generics or just write the code for each type
69 Unsafe & Cgo
unsafe package: raw memory operations that bypass the Go type system. The functions:
unsafe.Pointer— a pointer with no type, can be cast to/from any other pointer type oruintptr.unsafe.Sizeof(v)— size in bytes of a value (compile-time constant).unsafe.Alignof(v)— alignment requirement.unsafe.Offsetof(s.field)— byte offset of a field within its struct.unsafe.Add/unsafe.Slice/unsafe.SliceData(Go 1.17+) — typed pointer arithmetic.unsafe.String/unsafe.StringData(Go 1.20+) — zero-copy string ↔ []byte conversions.
import "C" with C code in the preceding comment block.
unsafe.Pointer looks like a C pointer but Go has no pointer arithmetic on raw pointers. To do arithmetic, you convert to uintptr, do math, then convert back. The compiler enforces specific patterns (documented in the unsafe package docs) and the GC may move values out from under you if you do it wrong — corrupting memory.Misuse causes:
- Memory corruption — silent at first, mysterious crashes later.
- GC confusion — the GC may free or relocate the wrong things.
- Race conditions invisible to the race detector.
- Non-portable code — assumptions about layout differ between architectures.
- Per-call cost: ~50–200 ns vs ~1 ns for a normal Go function call. 50–200× overhead.
- Goroutine scheduling: a goroutine making a Cgo call blocks its OS thread, so Go must spawn a new thread for other goroutines. This breaks Go's lightweight scheduling.
- Build complexity: requires a C compiler at build time, breaks fast cross-compilation, complicates Docker images.
- Memory ownership: Go GC and C malloc/free don't know about each other. Mixing them is error-prone.
- Stack switching: Cgo calls move from Go's small dynamic stacks to C's fixed stack — adds latency.
- Static linking is harder — Cgo introduces glibc dependencies that break
FROM scratchDocker images.
- vs C/C++: Native pointer arithmetic and unrestricted memory access. Go intentionally restricts this;
unsafeis the exception, not the rule. - vs Java JNI: JNI is similarly slow and complex; same trade-offs.
- vs Python C extensions: Comparable scope; Python's ctypes is a closer parallel to Cgo.
- vs Rust unsafe: Rust's
unsafeblocks are scoped and reviewable; Go'sunsafepackage is similar in spirit. - vs WebAssembly host calls: WASM has its own ABI; Cgo is OS-level FFI.
unsafein performance-critical libraries: zero-copystring↔[]byte, struct field offsets for serialization, custom memory layouts in databases/caches.- Cgo as a bridge to C libraries with no pure-Go alternative: sqlite3, libsodium, ffmpeg, OpenSSL (sometimes), GPU SDKs (CUDA, OpenGL), proprietary device drivers.
- Calling existing C code instead of rewriting it.
- Field-alignment optimization: use
unsafe.Sizeofand thefieldalignmentlinter to reduce struct padding.
go vet and govulncheck view their use as a red flag worth scrutinizing.
- Pure-Go libraries exist for many things you'd reach for C for (e.g.
modernc.org/sqlitefor sqlite3 in pure Go). - HTTP/RPC service wrapping the C library in another process.
- Protobuf/gRPC for cross-language interop without Cgo.
- WebAssembly for sandboxed C code in Go.
- Generics instead of
unsafefor type-erased containers.
// Struct padding — field order matters!
type Bad struct { a bool; b int64; c bool } // 24 bytes
type Good struct { b int64; a bool; c bool } // 16 bytes (33% less!)
// unsafe.Sizeof
fmt.Println(unsafe.Sizeof(int64(0))) // 8
fmt.Println(unsafe.Sizeof("")) // 16 (ptr + len)
// Go has NO pointer arithmetic (unlike C)
_ = Bad{}; _ = Good{} // suppress unused-type errors
- Avoid in app code — unsafe defeats type safety and the GC; misuse causes memory corruption
- strings ↔ []byte without copy — used in hot paths to skip the conversion allocation (Go 1.20+:
unsafe.String,unsafe.StringData) - Cgo for C libraries — wrapping libsodium, sqlite3, ffmpeg when no pure-Go alternative exists
- Cgo costs — slow Go↔C calls (~150ns each), breaks
go buildsimplicity, complicates cross-compilation - Struct field ordering — order from largest to smallest type to minimize padding (no unsafe needed; just careful struct design)
- Run
fieldalignment—go vet -vettool=$(which fieldalignment)reports wasted bytes from bad ordering
70 Design Patterns
- Functional Options: configure constructors with variadic
WithXfunctions.NewServer(WithPort(8080), WithTLS(cert)). Forward-compatible, self-documenting, no builder boilerplate. The defining Go pattern. - Singleton via
sync.Once: lazy-initialized package-level value.var get = sync.OnceValue(connect). - Strategy: an interface + multiple implementations swapped at runtime. Idiomatic in Go because interfaces are implicit and small.
- Decorator / Middleware: a function that wraps a Handler and returns a new Handler. The foundation of every Go HTTP framework.
- Pipeline: chained stages, each a goroutine, connected by channels. Idiomatic Go data processing.
- Pub/Sub: built from channels and a goroutine that fans out messages.
- Worker Pool: N workers reading from a shared jobs channel.
- Stateful Goroutine (Actor): one goroutine owns the state, others communicate via channels.
- Constructor Function:
NewX(args...) (*X, error)instead of exposing field initialization. - Embedding for composition: use struct/interface embedding instead of inheritance hierarchies.
- Iterator function:
func(yield func(T) bool)— Go 1.23+ formalized this withrange func. - Result type via multiple return:
(value, error)instead of an explicit Result<T, E>.
- Visitor: rarely needed because interfaces handle dispatch and type switches handle exhaustive matching.
- Abstract Factory: a function returning an interface usually suffices.
- Builder: replaced by Functional Options.
- Observer: implemented with channels instead of callback registries.
- Iterator: Go has
rangeand nowiter.Seq. - Template Method: use embedding or function fields instead of inheritance.
- Adapter: just a small wrapper struct or function.
- Command: a struct + a method, or just a closure.
- Most "creational" patterns collapse into "a function that returns a struct".
- vs Java: Java's pattern-heavy culture (singletons via private constructors, builders, factories of factories) becomes rare in Go because language features replace ceremony.
- vs Python: Python is duck-typed and uses decorators, context managers, and dynamic dispatch — patterns are again less ceremonious.
- vs Rust: Rust uses traits and enums (sum types) for many things Go does with interfaces and tagged unions.
- The "functional options" pattern is uniquely Go-flavored: it leverages variadic args, closures, and zero-value defaults to give you forward-compatible constructors with no builder boilerplate.
// Singleton (sync.Once)
var dbOnce sync.Once; var db *DB // package-level state
func GetDB() *DB {
dbOnce.Do(func() { db = connect() }) // connect only once
return db // always returns same instance
}
// Options Pattern (very common)
type Option func(*Server) // each option mutates Server
func WithPort(p int) Option { return func(s *Server) { s.port = p } } // set port
func WithTimeout(d time.Duration) Option { return func(s *Server) { s.timeout = d } } // set timeout
func NewServer(host string, opts ...Option) *Server {
s := &Server{host: host, port: 8080, timeout: 30*time.Second} // sensible defaults
for _, o := range opts { o(s) } // apply each option
return s
}
srv := NewServer("localhost", WithPort(3000)) // override only port
// Repository Pattern
type UserRepo interface { // abstract over storage
FindByID(ctx context.Context, id int) (*User, error) // read
Create(ctx context.Context, u *User) error // write
}
// Middleware Chain
type Middleware func(http.Handler) http.Handler // wraps a handler
func Chain(h http.Handler, mws ...Middleware) http.Handler {
for i := len(mws)-1; i >= 0; i-- { h = mws[i](h) } // apply in reverse for correct order
return h
}
_ = srv // suppress unused-var error
- Functional options — universal Go API pattern; used by every major library (gin, mongo-go-driver, gRPC client)
- Singleton via sync.Once — DB pools, HTTP clients, parsed templates — initialized lazily, exactly once
- Repository pattern — interface in service layer, concrete impl in infra layer; swap real DB for in-memory in tests
- Middleware chains — auth, logging, recovery, CORS — composed for every HTTP framework
- Strategy pattern via interfaces — different payment processors implementing a
Paymentinterface - Decorator — wrap an io.Reader/Writer with gzip, encryption, hashing — Go's interface composition makes this trivial
- Avoid Java patterns — Factory, AbstractFactory, Builder are usually overkill in Go; prefer simple functions
71 Production Project Structure
golang-standards/project-layout repo documents what most production Go services use. The standard top-level directories:
cmd/— one subdirectory per binary entry point, each containing amain.go.internal/— code that must not be importable by other modules. The compiler enforces this — any code underinternal/can only be imported by code within the same module's parent directory.pkg/(optional) — genuinely public reusable packages others can import.api/— OpenAPI specs, protobuf definitions, JSON schemas.configs/— config templates, default config files.scripts/— build, install, deploy scripts.deployments/— Dockerfiles, docker-compose, k8s manifests, terraform.migrations/— SQL migration files.test/— additional test apps and test data.docs/— design docs.
internal/:
config/— load env vars / YAML / flags into typed structs.domain/ormodel/— entities and value types with no dependencies.repository/orstore/— interfaces and implementations for data access.service/orusecase/— business logic, depending on repository interfaces.handler/ortransport/— HTTP/gRPC handlers, calling services.middleware/— cross-cutting handler wrappers.auth/,billing/,notification/— feature-bounded packages.
myapi/
├── go.mod
├── cmd/
│ ├── api/main.go # HTTP server
│ ├── worker/main.go # background job runner
│ └── migrate/main.go # DB migration CLI
└── internal/
├── db/ # shared by all binaries
├── domain/
└── service/
Each cmd/*/main.go is a tiny entry point that imports from internal/.
- vs Java
src/main/java/com/company/...: Go layouts are much flatter. No deep package hierarchies. - vs Rust: Rust uses
src/lib.rs+src/bin/*.rsfor libraries with multiple binaries. Comparable to Go'scmd/approach. - vs Node.js: Node has no convention — every project differs. Go's converged layout is much friendlier for newcomers.
- vs Python: Python uses
src/or package-name root; Go'scmd//internal//pkg/is more rigid. internal/is unique: compiler-enforced privacy across module boundaries, not just convention.
internal/ rule provides real encapsulation enforced by the compiler — you can't accidentally import another module's private code. The cmd/ + internal/ + pkg/ structure scales naturally from one binary to many.- Small CLIs and libraries — a flat layout (
main.go,foo.go,foo_test.go) is perfect. - One-off scripts — don't bother.
- Pure libraries — no
cmd/needed; the package itself is the entry point. - The Go team has publicly stated that the
project-layoutrepo is community-driven, not official. Don't treat it as gospel — adapt to your project's actual needs.
handler → service → repository → database
internal/domain has ZERO dependencies — pure Go structs and rules.
- cmd/ for entry points — one folder per binary (api, worker, migrate); each has its own main.go
- internal/ for app code — Go enforces that nothing outside the module can import these packages
- Don't over-layer small projects — for a 5-file CLI, just put everything in main.go; layers are for projects with multiple developers
- golang-standards/project-layout — popular reference, but controversial; treat as inspiration, not gospel
- Domain in the center — entities/business rules have no dependencies; outer layers depend on the center, never the reverse
- Test files alongside source —
user.go+user_test.goin the same package - Single go.mod per repo — multi-module repos add complexity; use only for libraries with versioned subpackages
72 Performance & Profiling
runtime/pprof— sampling profiler for CPU, heap, goroutines, blocks, and mutexes.net/http/pprof— exposes pprof profiles via HTTP endpoints (justimport _ "net/http/pprof").runtime/trace— fine-grained execution trace showing every goroutine, syscall, GC pause, and scheduler event.go tool pprof— analyzer with text, web UI, and flame graph output.go tool trace— visualizes trace files.- Test integration:
go test -cpuprofile=cpu.out -memprofile=mem.out -bench=. - Runtime hooks:
runtime.MemStats,runtime.NumGoroutine,runtime.GC.
- CPU profile — samples 100×/sec where the CPU is spending time. Best for finding hot functions.
- Heap profile (alloc / inuse) — what's allocating memory and what's currently held. Best for memory issues.
- Goroutine profile — stack traces of all live goroutines. Find leaks and stuck goroutines.
- Block profile — where goroutines spend time blocked on synchronization. Need
runtime.SetBlockProfileRate. - Mutex profile — contended mutexes. Need
runtime.SetMutexProfileFraction. - Allocs profile — total allocations over time, complementary to inuse.
- Add the import:
import _ "net/http/pprof"in your main package. - Run a debug server:
go http.ListenAndServe("localhost:6060", nil) - Capture a profile:
go tool pprof http://localhost:6060/debug/pprof/profile?seconds=30(CPU) or...?heap. - Analyze interactively:
(pprof) top10,(pprof) list FuncName,(pprof) web. - Or open the web UI:
go tool pprof -http=:8080 cpu.out— flame graphs, call graphs, source view.
- Preallocate slices:
make([]T, 0, expectedSize)avoids repeated re-grows. - Use
strings.Builderfor string concatenation in loops. - Avoid
interface{}in hot paths — boxing causes heap allocations. sync.Poolfor short-lived buffer reuse.- Order struct fields by size to reduce padding.
- Buffered I/O with
bufio. - Atomic ops instead of mutexes for single-value counters.
- Avoid reflection in hot paths — it's 10–100× slower.
- Use integer keys instead of strings in maps where possible.
- Compile-time check escape analysis:
go build -gcflags="-m".
- vs JVM profilers (YourKit, JProfiler): JVM profilers are powerful but proprietary or expensive. Go's pprof is free and built in.
- vs Linux perf: perf works on any binary but lacks Go-specific knowledge (goroutine stacks, GC events). Go's pprof understands the runtime.
- vs Python cProfile: Comparable; pprof is more powerful and produces flame graphs natively.
- vs Node.js clinic: clinic is a third-party tool; Go's is in stdlib.
- Always-on overhead: ~1–5% — safe to leave enabled in production behind a debug endpoint.
// pprof — built-in profiler
import _ "net/http/pprof" // blank import registers pprof routes
go http.ListenAndServe("localhost:6060", nil) // start debug server
// go tool pprof http://localhost:6060/debug/pprof/heap
// go tool pprof http://localhost:6060/debug/pprof/profile?seconds=30
// Key optimizations:
// 1. Pre-allocate: make([]int, 0, 1000)
// 2. strings.Builder for concatenation
// 3. sync.Pool for hot objects
// 4. Avoid interface{}/any (heap allocations)
// 5. go build -gcflags="-m" for escape analysis
// 6. Order struct fields by size (reduce padding)
// 7. Buffered I/O: bufio.NewWriterSize(f, 64*1024)
// 8. Race detector: go run -race main.go
- Profile before optimizing — pprof CPU + heap + goroutine profiles tell you the real bottleneck (often surprising)
- net/http/pprof in production — expose
/debug/pprof/on a private port for live profiling under real load - continuous profiling — Pyroscope/Polar Signals/Datadog Profiler collect always-on profiles per pod
- Allocations are usually the killer — reduce
allocs/opwith sync.Pool, pre-allocation, avoiding interface{} - Escape analysis —
go build -gcflags="-m"shows what allocates on heap; aim for stack allocation - Trace tool —
go tool tracevisualizes goroutine scheduling, GC pauses, syscall blocks - Don't tune what isn't slow — over-engineering performance hurts maintainability with no real-world benefit
73 Important Commands
go command-line tool — Go's single-binary, swiss-army-knife toolchain. One executable handles compiling, running, testing, formatting, vetting, dependency management, documentation, profiling, and cross-compilation. No Make, no Gradle, no npm scripts — just go <subcommand>.go run main.go— compile and execute in one shot. Your "run this file" button.go run .— run all.gofiles in the current directory.go build— compile to a binary in the current directory. Add-o nameto choose the output filename.go build ./...— build everything in the module recursively.go install— build and place in$GOPATH/bin(or$GOBIN).go fmt ./...— auto-format all code. Go has ONE style; debates are over.go vet ./...— static analysis: catches printf mismatches, unreachable code, bad struct tags, shadow vars, etc.go test ./...— run all tests.go test -race ./...— with race detector (REQUIRED in CI).go test -cover ./...— with code coverage.go test -bench=. -benchmem— run benchmarks with allocation reporting.
go mod init github.com/you/project— start a new module.go mod tidy— syncgo.modwith imports actually used. Add this after every import change.go get pkg@v1.2.3— add or update a specific version of a dependency.@latestfor newest.go mod download— pre-download deps without building (used in CI cache layers).go mod vendor— copy all deps into a localvendor/folder for offline builds.go mod verify— verify checksums matchgo.sum.go mod why pkg— explain why a package is in your module's dep graph.go list -m all— list all module dependencies.
go doc fmt.Println— show docs for a function in the terminal.go doc -all fmt— show all docs for a package.godoc -http=:6060— run a local docs server (legacy; modern is pkg.go.dev).go env— print the Go environment variables.go version— print the Go version.go tool pprof— analyze pprof profiles.go tool trace— analyze trace files.go tool cover -html=cover.out— render coverage as HTML.
GOOS and GOARCH to build for any platform from any platform — built into the toolchain, no Docker or VM needed:
- Linux ARM64:
GOOS=linux GOARCH=arm64 go build - Windows 64-bit:
GOOS=windows GOARCH=amd64 go build -o app.exe - macOS Intel:
GOOS=darwin GOARCH=amd64 go build - macOS Apple Silicon:
GOOS=darwin GOARCH=arm64 go build - WebAssembly:
GOOS=js GOARCH=wasm go build - List all targets:
go tool dist list
- vs Node.js: Node needs
node+npm+tsc+jest+eslint+prettier+nodemon, each with its own config file. - vs Python: Python needs
python+pip+venv+pytest+black+mypy. - vs Java:
javac+java+ Maven/Gradle + JUnit + Checkstyle. - vs Rust:
cargois comparable in scope — both ecosystems converged on a single tool. - vs C/C++: Make + CMake + various compilers and toolchains. Go is dramatically simpler.
go build ./..., go test -race ./..., go vet ./..., done. The single-binary toolchain is one of the biggest reasons Go projects ramp up so quickly — there's nothing to install, configure, or debate.The commands you'll actually use every day. Learn these and you're set.
| go run main.go | Compile + execute in one shot. Your "run this file" button. Also go run . for all files in dir. |
| go build -o myapp | Compile into a binary. Drop -o to use default name. This is what you ship to production. |
| go fmt ./... | Auto-format ALL your code. Go has ONE style — no debates. Run before every commit. |
| go vet ./... | Static analysis — catches bugs the compiler misses: wrong printf verbs, unreachable code, bad struct tags, etc. |
| go mod init github.com/you/project | Creates go.mod. Run once when starting a new project. This is your package.json. |
| go mod tidy | Syncs go.mod with your actual imports — adds missing deps, removes unused. Run after adding/removing imports. |
| go get github.com/pkg@v1.2.3 | Add or update a dependency. Use @latest for newest. This is your npm install. |
| go test ./... | Run ALL tests in all packages. The ./... means "recursively". This is the one you run most. |
| go test -v -run TestName | Run one specific test with verbose output. -run takes a regex. |
| go test -cover ./... | Shows coverage %. For detailed report: -coverprofile=c.out then go tool cover -html=c.out |
| go test -race ./... | Runs tests with the race detector. Catches concurrent data races. Use in CI — it's saved countless bugs. |
| go test -bench=. -benchmem | Run benchmarks. -benchmem shows allocations per op. Essential for perf work. |
| go build -gcflags="-m" | Escape analysis — tells you what allocates on heap vs stack. Add -m -m for verbose. |
| go run -race main.go | Run your app with race detection. Slower but catches goroutine data races at runtime. |
| go tool pprof http://localhost:6060/debug/pprof/heap | Analyze memory profile of a running app (needs import _ "net/http/pprof"). |
| GOOS=linux GOARCH=amd64 go build -o app | Build for Linux from your Mac. That's it. No Docker needed for compilation. Change GOOS/GOARCH for any target. |
| go build -ldflags="-s -w" | Smaller binary — strips debug symbols. Typical Go binary: 10-15MB, with this: 6-8MB. |
| go install | Build + put the binary in $GOPATH/bin. For CLI tools you want available globally. |
| go clean -cache | Nuke the build cache. Fixes weird "it should work but doesn't" issues. |
| go clean -testcache | Clear cached test results. Use when tests pass but shouldn't (or vice versa). |
| go env | Print all Go env vars. Useful when GOPATH, GOROOT, or GOPROXY seems off. |
| go doc fmt.Sprintf | Quick docs lookup from terminal. Faster than googling. |
- Pre-commit hooks — chain
go fmt ./...,go vet ./...,golangci-lint run,go test ./... - CI pipeline — always run
-racetests,-covercoverage, dependency vulnerability scan (govulncheck ./...) - Cross-compile in CI — build Linux/macOS/Windows binaries from one machine (no per-OS runners needed)
- go mod tidy in CI — fail the build if go.mod is out of sync (
go mod tidy && git diff --exit-code go.mod go.sum) - Reproducible builds — use
-trimpathand-buildvcsfor byte-identical binaries across hosts - golangci-lint — single binary that runs 50+ linters; the de facto standard for Go quality gates
74 Interview Questions
- Language fundamentals: slices vs arrays, value vs pointer receivers, interface satisfaction, struct embedding, defer order, init functions.
- Memory model: escape analysis, stack vs heap, value vs reference semantics, GC behavior, struct padding.
- Concurrency: goroutines and channels, select, mutexes, atomics, race conditions, deadlocks, context propagation, worker pools, errgroups.
- Runtime internals: the M:N scheduler (G/M/P), preemption, GC (tri-color mark-sweep), GOMAXPROCS, channel implementation.
- Error handling: wrapping with
%w,errors.Is/As, sentinel vs typed errors, panic vs error. - Design judgment: when to use generics, when to use channels vs mutexes, when to use interfaces, when to use embedding vs composition.
- Standard library: common packages, surprising behaviors, performance characteristics.
- Tooling: testing patterns, benchmarking, profiling, vet, race detector, fuzzing.
- Junior: language fundamentals; expected from anyone with 6+ months of Go.
- Mid: concurrency, interfaces, error wrapping; expected from someone shipping Go to production.
- Senior: runtime internals (GMP, GC, escape analysis), error patterns, context propagation, advanced testing.
- Staff/VP: system design, profiling methodology, project structure trade-offs, library design.
- Heavy on concurrency reasoning: deadlock spotting, goroutine leak detection, channel patterns, race condition examples. This is THE Go-specific area.
- Less algorithm trivia: compared to Java or JS interviews, you'll see fewer "implement Quicksort in 5 minutes" puzzles and more "given this code, what does it print and why" questions about scheduling, recover, and closure capture.
- Practical over theoretical: "How would you make this safer?" and "What's the bug?" rather than "Explain monads."
- Whiteboard concurrency: implementing a worker pool, bounded queue, or rate limiter from scratch is a frequent live exercise.
- Code-reading questions are more common than code-writing — Go's small syntax surface makes "spot the bug" interviews productive.
- Don't memorize answers — interviewers ask follow-ups; understand the "why" deeply.
- Practice live coding in a plain editor, not your IDE — you may not have autocomplete.
- Always check error returns in interview code — even pseudo-code. Interviewers notice.
- Talk through your thinking — interviewers care about reasoning more than the final answer.
- Ask clarifying questions — "Should this handle concurrent access?" reveals depth.
- Know the Go-specific footguns: loop variable capture, nil interface vs nil pointer, slice mutation through subslice, map iteration order.
Junior Level
var x int is an explicit declaration — can be used at package level or inside functions. x := 0 is short variable declaration — can ONLY be used inside functions. Both initialize x to 0, but := also infers the type.
var x int // explicit type, zero value (0)
x := 0 // inferred as int, value 0
x := 0.0 // inferred as float64!
Go's philosophy is to keep codebases clean. Unused variables and imports are dead code — they confuse readers, slow down compilation, and often indicate bugs. The compiler enforces this as a hard error, not a warning, to prevent code rot.
Use _ (blank identifier) when you intentionally want to discard a value:
_, err := doSomething() // discard first return value
Every type has a zero value: int → 0, string → "", bool → false, pointers/slices/maps → nil. This means variables are always initialized — no undefined behavior. Many Go types are designed to be useful at their zero value (e.g., sync.Mutex{}, bytes.Buffer{}).
Array: fixed size, size is part of the type ([3]int ≠ [5]int), value type (copying copies all elements).
Slice: dynamic, reference type (header: pointer + length + capacity), backed by an array. Use slices 99% of the time.
arr := [3]int{1,2,3} // array — fixed
slc := []int{1,2,3} // slice — dynamic
Go treats errors as values, not exceptions. Functions return an error as the last return value. You check it explicitly with if err != nil. This is intentional — it makes error handling visible and forces you to think about every failure point. No hidden control flow.
f, err := os.Open("file.txt")
if err != nil {
return fmt.Errorf("open file: %w", err) // wrap & propagate
}
defer f.Close()
defer schedules a function call to run when the surrounding function returns. Multiple defers execute in LIFO (last-in-first-out) order. Common uses: closing files, unlocking mutexes, flushing buffers.
mu.Lock()
defer mu.Unlock() // guaranteed even if panic
Mid Level
Go allocates a new, larger underlying array (roughly 2x capacity for small slices, 1.25x for large), copies existing elements, then appends. This is why append returns a new slice — the pointer may have changed. Always: s = append(s, x).
If two slices shared the same array and one grows past capacity, they now point to different arrays — mutations no longer affect each other.
Value receiver (r Rect): gets a copy. Use for small, read-only structs.
Pointer receiver (r *Rect): gets a pointer, can mutate. Use when: you need to modify the struct, the struct is large, or for consistency (if one method needs pointer, use pointer for all).
Important: a value receiver can be called on a pointer, and vice versa — Go auto-converts. But an interface is satisfied differently: Shape implemented by *Rect means only *Rect (not Rect) satisfies it.
Go interfaces are implicit — a type satisfies an interface by implementing its methods, without declaring it. This enables decoupling without import dependencies. Internally, an interface value is a two-word pair: (type, value).
Best practices: keep interfaces small (1-3 methods), define them where they're used (not where implemented), and "accept interfaces, return structs."
A goroutine is a lightweight thread managed by the Go runtime. It starts with ~2KB stack (vs ~1-8MB for OS threads), is multiplexed onto OS threads by the Go scheduler (M:N scheduling), and context-switches faster since it doesn't need a syscall. You can easily run millions of goroutines.
The Go scheduler uses a work-stealing algorithm with per-P (processor) run queues for efficiency.
Unbuffered make(chan int): synchronous — sender blocks until receiver is ready. Use for synchronization / guaranteed handoff.
Buffered make(chan int, 10): sender only blocks when full. Use for: decoupling producer/consumer speeds, rate limiting, batch processing, semaphores.
Rule of thumb: start unbuffered, add buffer only when you need to decouple timing.
select blocks until one of its cases can proceed. If multiple are ready, it picks one randomly (fair scheduling). Common patterns: timeout with time.After, non-blocking with default, done channel for cancellation.
select {
case v := <-dataCh: process(v)
case <-ctx.Done(): return
case <-time.After(5*time.Second): return errTimeout
}
Senior Level
G = Goroutine (the task). M = Machine/OS thread (the executor). P = Processor (the scheduler context, holds run queue). Default GOMAXPROCS = number of CPU cores.
Each P has a local run queue of Gs. An M must acquire a P to execute Gs. When a goroutine blocks (syscall, channel, etc.), the M releases its P so another M can pick it up. Work-stealing: idle Ps steal Gs from busy Ps' queues.
This M:N model gives goroutine-level concurrency with OS-level parallelism.
Go uses a concurrent, tri-color mark-and-sweep GC. It runs concurrently with your program (mostly). Three phases: Mark Setup (STW, brief), Marking (concurrent), Mark Termination (STW, brief).
Tri-color: White (unmarked/garbage), Grey (reachable, not fully scanned), Black (reachable, fully scanned). Write barrier ensures correctness during concurrent marking.
Tuning: GOGC=100 (default) — triggers GC when heap doubles. Lower = more frequent GC, less memory. GOMEMLIMIT (Go 1.19+) sets a soft memory limit. Use GODEBUG=gctrace=1 to observe.
The compiler analyzes whether a variable can stay on the stack or must "escape" to the heap. Stack allocation is free (just move stack pointer), heap allocation requires GC.
Variables escape when: returned as pointer, stored in interface, captured by closure that outlives function, too large for stack.
Check with: go build -gcflags="-m". To optimize: return values not pointers when small, avoid unnecessary interface{}, pre-allocate slices.
Goroutine leaks happen when a goroutine blocks forever (waiting on channel, mutex, etc.). Prevention:
- Always use
context.Contextfor cancellation - Ensure channels are closed or have done signals
- Use buffered channels when producers might exit before consumers
- Set timeouts on operations
- In tests, use
goleak(uber-go/goleak) to detect leaks
ctx, cancel := context.WithTimeout(ctx, 5*time.Second)
defer cancel()
select {
case r := <-work(ctx): // ...
case <-ctx.Done(): return ctx.Err()
}
errors.Is(err, target) checks if any error in the chain matches a specific value (for sentinel errors like ErrNotFound). Uses == comparison or custom Is() method.
errors.As(err, &target) checks if any error in the chain matches a specific type and extracts it (for custom error structs). Fills the target variable with the matched error.
if errors.Is(err, sql.ErrNoRows) { /* value match */ }
var ve *ValidationError
if errors.As(err, &ve) { /* type match, ve is populated */ }
Context carries: cancellation signals, deadlines/timeouts, and request-scoped values across API boundaries and goroutines. It forms a tree — cancelling a parent cancels all children.
Critical in servers: when a client disconnects, the request context is cancelled, which should propagate to all goroutines doing work for that request (DB queries, HTTP calls, etc.), preventing wasted resources.
Rules: first param, don't store in struct, always defer cancel().
Staff / VP Level
Use the pipeline pattern: chain stages connected by channels. Each stage is a goroutine (or pool) that reads from input channel, processes, writes to output channel.
Fan-out: multiple goroutines reading from same channel (parallel processing). Fan-in: merge multiple channels into one.
Key decisions: buffer sizes (tune to balance memory vs throughput), backpressure (bounded channels), cancellation (context), error handling (dedicated error channel or errgroup), graceful shutdown (close input, drain pipeline).
Use golang.org/x/sync/errgroup for managing groups of goroutines with error propagation.
CPUs read memory in word-sized chunks. Fields must be aligned to their size (int64 to 8-byte boundary). The compiler inserts padding bytes between misaligned fields.
type Bad struct { a bool; b int64; c bool } // 24 bytes (7+7 padding)
type Good struct { b int64; a bool; c bool } // 16 bytes (6 padding)
Order fields from largest to smallest. For hot structs in slices, this saves cache lines and memory. Use fieldalignment linter or unsafe.Sizeof to check.
- Measure first: add pprof endpoints, collect CPU + heap profiles under realistic load
- CPU profile:
go tool pprof→ find hot functions, optimize algorithms - Memory profile: heap profile → find allocation-heavy paths, reduce with sync.Pool, pre-allocation, avoid interface{}
- Escape analysis:
-gcflags="-m"→ keep hot-path allocations on stack - Benchmarks:
go test -bench -benchmembefore and after changes - Trace:
go tool tracefor goroutine scheduling, GC pauses, latency analysis - GC tuning: adjust GOGC, set GOMEMLIMIT, observe with gctrace
Golden rule: don't optimize without a profile. Measure, change, measure again.
internal/ is enforced by the Go compiler — packages under it can only be imported by code rooted at the parent of internal. This creates a hard visibility boundary.
pkg/ is a convention (not enforced) for packages intended to be importable by external projects.
Default to internal/ — making something public is an API commitment. Only move to pkg/ when you explicitly want external consumers. Many modern Go projects skip pkg/ entirely and put public packages at the root.
- Junior — language fundamentals; expected from anyone with 6+ months of Go
- Mid — concurrency, interfaces, error wrapping; expected from someone shipping Go to production
- Senior — runtime internals (GMP, GC, escape analysis), error patterns, context propagation
- Staff/VP — system design, profiling methodology, project structure trade-offs
- Don't memorize answers — interviewers ask follow-ups; understand the "why" deeply
- Whiteboard concurrency primitives — implementing a worker pool / bounded queue from scratch is a common live exercise
75 Ready for Gin Framework
net/http. It doesn't replace the standard library; it wraps it with conveniences:
- Router with parameter matching:
r.GET("/users/:id", ...) - Middleware chaining:
r.Use(Logger(), Recovery(), Auth()) - JSON binding and validation: auto-decode request bodies into structs.
- Structured response helpers:
c.JSON(200, gin.H{"key": "value"}) - Context-per-request API:
c *gin.Contextwraps both request and response with helpers. - Built-in panic recovery middleware.
- Route groups for organizing routes by prefix or middleware.
- Server-sent events, multipart forms, file serving, and more.
http.HandleFunc("/users/", func(w http.ResponseWriter, r *http.Request) {
// Manually parse path, headers, JSON body, write response
parts := strings.Split(r.URL.Path, "/")
// ...
})
Gin:
r := gin.Default()
r.GET("/users/:id", func(c *gin.Context) {
id := c.Param("id")
c.JSON(200, gin.H{"id": id})
})
r.Run(":8080")
Gin's value is the boilerplate it removes, not anything it adds underneath — every request is still a goroutine running on net/http.
- Gin: the most popular Go web framework. Largest middleware ecosystem. Built on
net/http. - Echo: very similar feature set to Gin, slightly different API style. Also built on
net/http. - Chi: thinner, more stdlib-compatible. Returns plain
http.Handlers — drop-in compatible with anything in the Go ecosystem. Favored by purists. - Fiber: Express.js-style API. Built on fasthttp, not
net/http— faster but incompatible with the standard ecosystem. - Buffalo: full-stack framework with code generation; less popular.
- Plain
net/http+ Go 1.22 routing: the newhttp.ServeMuxwith method/path patterns is "good enough" for many APIs without any framework.
- Most popular = the most learning resources, examples, blog posts, and Stack Overflow answers.
- Largest middleware ecosystem — auth (JWT), CORS, rate limiting, request ID, observability, gzip, prometheus metrics — all available off the shelf.
- Familiar to developers coming from Express (Node), Flask (Python), or Sinatra (Ruby).
- Built on stdlib — no proprietary HTTP server, integrates with anything that takes
http.Handler. - Active maintenance and a huge GitHub star count.
- Production-proven at scale.
net/http, Gin removes the boilerplate without hiding what's happening underneath. Use Gin when:
- You're building a REST or JSON API.
- You want a familiar Express-like API.
- You want a large existing middleware ecosystem.
- Your team is mixed-experience and a popular framework reduces ramp-up.
net/http or Chi) when:
- You want to stay close to stdlib and avoid lock-in.
- Your API is small enough that the standard library handles it.
- You need to integrate with non-Gin middleware that expects
http.Handler. - You value minimal dependencies in a security-sensitive context.
Every concept maps to what you learned above.
package main // executable package
import (
"net/http" // for status codes like StatusOK
"strconv" // for Atoi (string to int)
"github.com/gin-gonic/gin" // Gin web framework
)
type User struct {
ID int `json:"id"`
Name string `json:"name" binding:"required"` // required in request body
Email string `json:"email" binding:"required,email"` // must be valid email
}
var users = make(map[int]User) // in-memory store
var nextID = 1 // auto-increment ID counter
func main() {
r := gin.Default() // router with Logger + Recovery
r.Use(authMiddleware()) // apply auth to all routes
api := r.Group("/api/v1") // prefix all routes with /api/v1
{
api.GET("/users", getUsers) // list all users
api.GET("/users/:id", getUserByID) // :id is a URL param
api.POST("/users", createUser) // create new user
}
r.Run(":8080") // start server on port 8080
}
func getUsers(c *gin.Context) {
list := make([]User, 0) // ensure JSON array not null
for _, u := range users { list = append(list, u) } // collect all users
c.JSON(http.StatusOK, gin.H{"data": list}) // gin.H is map[string]any
}
func getUserByID(c *gin.Context) {
id, _ := strconv.Atoi(c.Param("id")) // Param extracts :id from URL
u, ok := users[id] // look up user by ID
if !ok { c.JSON(404, gin.H{"error": "not found"}); return } // not found
c.JSON(http.StatusOK, u) // return user as JSON
}
func createUser(c *gin.Context) {
var u User // target for JSON binding
if err := c.ShouldBindJSON(&u); err != nil { // validate & decode body
c.JSON(400, gin.H{"error": err.Error()}); return // bad request
}
u.ID = nextID; nextID++ // assign auto-increment ID
users[u.ID] = u // store the new user
c.JSON(http.StatusCreated, u) // 201 Created response
}
func authMiddleware() gin.HandlerFunc { // returns a Gin handler
return func(c *gin.Context) {
token := c.GetHeader("Authorization") // read auth header
if token == "" { // no token provided
c.AbortWithStatusJSON(401, gin.H{"error": "unauthorized"}) // stop chain
return
}
c.Set("userID", "123") // attach data to request context
c.Next() // proceed to next handler
}
}
Concept Mapping
| What You Learned | Used in Gin |
|---|---|
| Structs + JSON tags | Request/Response models, ShouldBindJSON |
| Interfaces | Gin's Handler, custom middleware |
| Error handling | Every handler checks errors |
| Closures | Middleware pattern |
| Maps | Route params, gin.H{} |
| Pointers | *gin.Context everywhere |
| Context | Request context, timeouts |
| Goroutines | Each request = its own goroutine |
| Options pattern | Server / middleware configuration |
| Testing | httptest for handler tests |
- REST APIs — most popular Go web framework;
r.Group("/api/v1")for versioning, middleware groups for auth - Microservices — Gin + gRPC: HTTP gateway in front of internal gRPC services
- Webhook receivers — Gin handlers parse JSON, verify HMAC signatures, dispatch jobs
- Admin dashboards — Gin + html/template for server-rendered admin panels
- File uploads —
c.FormFile+c.SaveUploadedFilefor multipart handling - JWT auth middleware — verify token in middleware, attach
userIDto context, handlers retrieve it - Validation via binding tags —
binding:"required,email,min=8"rejects bad input before reaching the handler - Alternatives — Echo (similar API), Fiber (Express-like), chi (closer to net/http)
- Practice data structures — implement from memory
- Build a REST API with
net/http— Go Dev Guide covers this end-to-end - Then move to Gin — it's syntactic sugar over what you know
- Add PostgreSQL (
sqlxorGORM) - Add JWT authentication — covered in Go Dev Guide
- Learn gRPC & Protobuf — Go Dev Guide covers server, client, streaming & gateway
- Deploy with Docker
REST APIs, Middleware, gRPC, TLS, JWT, Password Hashing — all with production code.
Built for mastery. Now go build something.