Database field guide · 16
Partitioning and Sharding
Pruning, placement, and global constraints
Design range, list, and hash boundaries that improve lifecycle and locality without sacrificing SQL semantics.
Franck Pachot7 chapterspartitioning · pruning · sharding · placement
Build the mental model before choosing the mechanism.
This minibook was AI-generated from Franck Pachot's archived blog posts. Links to the original articles are included for source context and verification.
01
Partition for a boundary
Partitioning is valuable when a boundary supports pruning, lifecycle operations, locality, or independent maintenance. Merely splitting a table does not reduce the work of an unprunable query.
- Choose a key present in important predicates.
- Keep partition count operationally bounded.
- Separate retention needs from distribution needs.
02
Pruning is optimizer proof
Static pruning uses known values at planning time; runtime pruning uses parameters or join values during execution. Expressions and implicit casts can hide the boundary.
- Inspect partitions actually scanned.
- Bind variables can delay pruning decisions.
- Global and partition statistics both affect costing.
03
Range and hash solve different problems
Range partitions preserve locality and permit detach-by-time operations. Hash distributes keys broadly but gives up adjacent-key locality.
- Monotonic keys can hotspot one range.
- Hash does not replace a queryable business key.
- Composite strategies can separate routing from local order.
04
Constraints may be global
Uniqueness and foreign keys become harder when the constrained key omits the partition key. Engines use global indexes, distributed transactions, or restrictions to preserve semantics.
- Declare the real invariant before choosing local indexes.
- A partition-local unique key has narrower meaning.
- Test attach and detach validation costs.
05
Distributed tablets move
A distributed shard is also a replication and balancing unit. Splitting and movement change physical placement while SQL identity should remain stable.
- Tablet count affects metadata and consensus overhead.
- Co-location reduces some RPCs but concentrates load.
- Observe skew in bytes, requests, and leaders.
06
Queries merge local answers
Top-N, aggregation, and joins across partitions require a global merge unless predicates prove only one partition matters.
- Push limits only when semantics allow it.
- Preserve a total order across partition boundaries.
- Measure rows read from every child, not only final output.
07
Field manual
Concrete mechanics, diagnostic evidence, and executable patterns to carry into a real system.
01
Create lifecycle-oriented ranges
A default partition catches unexpected values but can hide routing defects. Use half-open boundaries and automate creation before the boundary arrives.
CREATE TABLE event (...) PARTITION BY RANGE (event_time);
CREATE TABLE event_2026_08 PARTITION OF event
FOR VALUES FROM ('2026-08-01') TO ('2026-09-01');
02
Verify pruning
EXPLAIN must show which children are scanned and which are removed. Parameterized statements may prune at initialization or execution rather than planning.
EXPLAIN (ANALYZE, BUFFERS, VERBOSE)
SELECT * FROM event
WHERE event_time >= $1 AND event_time < $2;
-- Inspect Subplans Removed and loops for every partition.
03
Preserve uniqueness semantics
PostgreSQL requires a partitioned unique constraint to include all partition key columns because no global index arbitrates duplicate keys across children.
ALTER TABLE reservation ADD CONSTRAINT reservation_uk
UNIQUE (tenant_id, reservation_id);
-- If reservation_id alone is globally unique, enforce it elsewhere or redesign.
04
Expose distributed placement
YugabyteDB hash keys distribute writes while following range columns retain order within a tablet keyspace. Explain distribution as part of schema review.
CREATE TABLE reading (
device_id bigint, observed_at timestamptz, value numeric,
PRIMARY KEY ((device_id) HASH, observed_at DESC)
) SPLIT INTO 24 TABLETS;
08
Source articles
Optional deep dives with the complete experiments and product-version context behind this guide.