Paste Bin

i. Requirements

Functional

• Create a paste: accepts text or code up to 10 MB, returns a short unique URL
• Retrieve a paste by its short URL: serve the raw content
• Optional expiration: never, 10 min, 1 hour, 1 day, 1 week, 1 month, or custom
• Optional: syntax highlighting by language (client-side is fine)
• Optional: password-protected pastes (read requires correct passphrase)
• Optional: custom alias (vanity slug), burn-after-read mode
• Optional: fork / clone an existing paste

Non-Functional

• Durability — content must not be lost; pastes are the product
• Availability — reads at 99.99%, writes at 99.9%
• Low read latency — p99 < 50 ms (content may be large; time-to-first-byte is the target)
• Scalable storage — total content grows without bound; storage cost must be proportional to actual bytes
• Read-heavy at ~10:1, skewed: popular pastes are fetched thousands of times, most are fetched once

Out of Scope

• Real-time collaboration (Google Docs–style concurrent editing)
• Full user account system, billing, teams
• Server-side syntax highlighting at render time (offload to the client)
• Diff / version history beyond forking

ii. Capacity Estimates

Per-tier sizing — back of the envelope

The read QPS looks modest (3,500) but the byte rate is not. CDN absorption is the single biggest lever, especially since popular pastes are fetched repeatedly.

iii. High-Level Design

Client
  │
  ▼
[Geo-DNS / Anycast]
  │
  ▼
[CDN Edge PoP]  ──hit (public paste)──▶  response (content from edge cache)
  │ miss
  ▼
[Regional Load Balancer]
  │                          │
  ▼                          ▼
[Read Service]          [Write Service]
  │                          │
  ├─▶ [Redis L2]             ├─▶ [ID Generator (base62)]
  │    hit → return          ├─▶ [Object Storage: S3/GCS]  ──▶  content bytes
  │    miss ↓                └─▶ [Metadata DB: Cassandra]  ──▶  row (id, ttl, key, …)
  ├─▶ [Metadata DB]
  │    get content_key
  │
  └─▶ [Object Storage: S3/GCS]  ──▶  stream content bytes to client

[Expiry Worker] ──reads TTL queue──▶ deletes S3 object + DB row

The core split: metadata (paste ID, expiry, owner, content_key, size, language) lives in Cassandra; content bytes live in object storage (S3). The read service stitches them together. This split is the central design decision — everything downstream follows from it.

For small pastes (<4 KB), content is stored inline in the Cassandra row and the S3 hop is skipped entirely. The 4 KB threshold keeps the DB row under the Cassandra recommended limit for inline blobs while eliminating the object storage round-trip for the majority of pastes (code snippets, config fragments, short logs).

iv. Key Design Decisions

ID scheme

Same base62 approach as the URL shortener — a 7-character slug gives 62⁷ ≈ 3.5 trillion unique IDs, enough for any realistic horizon. ID generation uses a distributed counter with ZooKeeper-coordinated ranges: each write pod claims a range of 1 million IDs, burns through them locally (no coordination per write), then claims another. Collision is structurally impossible within a range.

For custom aliases, the write path does a Cassandra LWT (lightweight transaction) INSERT IF NOT EXISTS on the alias. Races are rare; LWT handles them correctly without distributed locks.

The inline vs. object storage split

Database schema (Cassandra)

Cassandra is chosen over Postgres for the same reasons as the URL shortener: the access pattern is a pure point lookup by paste_id, writes are heavy (10M/day × RF=3 = 30M physical writes), and the 10-year storage horizon demands horizontal scaling. A Postgres primary + replicas would work for the first two years; migrate at ~500M rows.

-- Primary table: all reads go here
CREATE TABLE pastes (
  paste_id      text PRIMARY KEY,       -- base62 slug, e.g. "aB3kZ9m"
  created_at    timestamp,
  expires_at    timestamp,              -- null = never expires
  owner_id      text,                   -- null for anonymous
  language      text,                   -- "python", "sql", null
  title         text,
  size_bytes    int,
  content_type  text,                   -- "inline" | "s3"
  content_key   text,                   -- S3 key if content_type = "s3"
  content_inline blob,                  -- raw bytes if content_type = "inline"
  password_hash text,                   -- bcrypt hash; null = public
  burn_on_read  boolean,
  view_count    counter                 -- approximate; see note
) WITH default_time_to_live = 0        -- TTL managed at app layer, not Cassandra native
  AND compaction = { 'class': 'LeveledCompactionStrategy' };

-- Expiry index: drives the cleanup worker
CREATE TABLE pastes_by_expiry (
  expiry_bucket text,    -- "2026-05-28T14" (hour granularity)
  expires_at    timestamp,
  paste_id      text,
  PRIMARY KEY (expiry_bucket, expires_at, paste_id)
) WITH CLUSTERING ORDER BY (expires_at ASC, paste_id ASC);

-- Owner index: "my pastes" lookup (secondary access pattern)
CREATE TABLE pastes_by_owner (
  owner_id   text,
  created_at timestamp,
  paste_id   text,
  title      text,
  PRIMARY KEY (owner_id, created_at, paste_id)
) WITH CLUSTERING ORDER BY (created_at DESC, paste_id ASC);

Notes on schema decisions: Cassandra native TTL is not used for the main table because it fires on a per-row clock that cannot be changed after write. If an owner extends a paste's expiry, a native TTL would silently delete it anyway. App-layer expiry (checked at read time + async worker deletion) is more flexible. The view_count is a Cassandra counter column — approximate under concurrent increments, but accurate enough for analytics and "popular" ranking.

Caching strategy

Negative caching: A missing paste_id (404) is cached in Redis for 60s with a sentinel value. Without this, a scan-and-fetch loop against random IDs would hammer Cassandra on every miss.

Stampede protection: The read service uses Go's singleflight in-process for concurrent identical requests (a popular paste shared via social media sees a burst of simultaneous first-loads). At Redis level, only one request holds a distributed mutex to rebuild the cache entry; others wait with a 200ms timeout, then fall through to S3 directly if the mutex holder times out.

Pre-signed S3 URLs: For large pastes, Redis caches a pre-signed S3 GET URL (valid 15 minutes) rather than the bytes themselves. The client is redirected to S3 directly, offloading bandwidth from the app tier entirely. This is the right trade-off when paste size is large enough that Redis memory cost exceeds S3 GET cost.

v. Deep Dives

Expiration pipeline

Expiry is the feature that quietly makes the system hard. A paste with expires_at = T must stop being readable at T, and its bytes must eventually be deleted from S3 to reclaim storage. There are three components:

1. Read-time enforcement: Every read checks expires_at against now() in the metadata returned from Cassandra or Redis. Expired pastes return 404 immediately. This is the correctness layer — it fires even if the async cleanup worker is lagging.
2. Expiry worker (async cleanup): A background service reads pastes_by_expiry in hour-bucket order. For each expired paste it: (a) deletes the S3 object if content_type = "s3", (b) deletes the Cassandra rows across all three tables, (c) invalidates CDN cache via purge API, (d) deletes from Redis. Order matters: S3 first, then DB. If the worker crashes between S3 delete and DB delete, a subsequent read gets a 404 from the DB (expired) or a 404 from S3 (object gone) — either way correct. The reverse order (DB then S3) would leave orphaned bytes on S3 indefinitely.
3. S3 lifecycle rules as backstop: A lifecycle rule on the S3 bucket tags objects at creation with expires_at and sets a lifecycle policy to delete objects after their tag date. This is a belt-and-suspenders measure — it catches anything the worker missed due to a multi-day outage.

Burn-after-read

A paste with burn_on_read = true is deleted immediately after the first successful read. The implementation requires care: reading and deleting must be atomic-ish. The read service: (1) fetches metadata from DB, (2) checks burn flag, (3) if set, issues a Cassandra DELETE before returning the content to the client, (4) purges CDN and Redis. The window between step 3 and the CDN purge propagating is typically < 5 seconds. If two readers race, one wins and the other gets a 404 — acceptable behavior, documented in the product.

Password-protected pastes

The password hash (bcrypt, cost factor 12) is stored in the Cassandra row. On read, the client submits the passphrase in a POST body (never in the URL — URLs end up in logs). The read service bcrypt-compares and returns 401 on mismatch. Password-protected pastes are excluded from CDN and Redis caching (served from origin only). Pre-signed S3 URLs are not used for password-protected pastes — the bytes must not be directly reachable without auth.

Syntax highlighting

Done entirely client-side using a library (Prism.js or highlight.js loaded from CDN). The server stores language as a metadata field; the client uses it to trigger the right grammar. This keeps the read path simple and stateless — no server-side rendering, no compute for highlighting. The trade-off is that curl-ing a paste returns raw text, which is almost always what developers want anyway.

Forking a paste

Fork = create a new paste with the same content but a new ID and new owner. The write service copies content from the original: for inline pastes, reads the blob from the parent row and writes it inline in the new row; for S3 pastes, issues an S3 server-side copy (no data moves across the network — S3 handles it internally, billing at copy cost not data transfer). The fork then proceeds as a normal write. This is cheap and correct.

vi. Bottlenecks by Tier

vii. Hot Keys / Skew / Pathological Data

The skew profile here is more extreme than in the URL shortener. A paste shared in a viral tweet or a Hacker News "Show HN" comment can go from 0 to 50,000 reads in 60 seconds. The paste ID becomes a hot key across every tier simultaneously: CDN, Redis, S3, and Cassandra all see it at once.

Mitigations

• CDN as the first wall: Public pastes are aggressively cached at the CDN edge. A viral paste that reaches CDN cache serves infinitely without touching origin. Set Cache-Control: public, max-age=3600, stale-while-revalidate=60 on public permanent pastes. The stale-while-revalidate window means the CDN serves from cache while asynchronously refreshing, so origin never sees thundering herd on TTL expiry.
• In-process LRU as the second wall: Each app pod maintains an LRU of the last 1,000 paste responses in memory (~10 MB at 10 KB average). A pod that has already seen this paste serves subsequent requests without touching Redis or S3.
• Pre-signed S3 URLs redirect clients directly to S3: For large pastes, the app returns a 307 redirect to a pre-signed S3 URL. Bandwidth shifts from app pods + Redis to S3 directly, distributing the load across S3's own CDN. This is the key architectural advantage of object storage for large content.
• Hot-paste detection and adaptive caching: A background counter (probabilistic sampling via reservoir sampling at 1%) identifies paste IDs with >1,000 requests/minute. These are flagged for: (a) extended CDN TTL via purge-and-rewrite, (b) promotion to in-process L1 on all pods via a lightweight pub-sub event, (c) S3 pre-signed URL caching in Redis for maximum duration. Detection latency is ~60 seconds — short enough to catch viral spikes before they saturate the origin.

viii. Multi-Region Architecture

Routing layer

Anycast geo-DNS routes readers to the nearest region. Pastebin reads are entirely region-local: the CDN serves from edge; cache misses hit the regional read service and regional Cassandra replica. No cross-region hop on the read path.

Writes are slightly different. Anonymous pastes are created in the nearest region. Authenticated user pastes are created in the user's "home region" (determined at account creation) to simplify the pastes_by_owner lookup. If a user in Singapore creates an account, their pastes live in the AP region's Cassandra. Reads from the US get a cross-region hop for cache misses — acceptable for a write-once-read-many workload where the vast majority of reads are already CDN hits.

What is region-local vs. globally coordinated

RPO / RTO matrix

ix. Failure Modes & Mitigations

Infrastructure failures

Data-path pathologies

Operational / deployment failures

x. Security & Abuse

Threat model

Takedown propagation flow

Detection → flag in DB (status = 'tombstoned') → read service serves 451 (legal / abuse takedown) → purge CDN edge cache via API (propagates to all PoPs in < 10 seconds) → delete Redis cache → S3 delete or legal hold (depending on reason) → audit log entry. Total time from detection to edge cache cleared: < 60 seconds.

xi. Observability & SLOs

SLI targets (28-day rolling window)

Golden signals — per service

• Traffic: RPS at each tier (CDN, read service, Redis, Cassandra, S3); break out by HTTP status and by content_type (inline vs. S3 pastes)
• Errors: 5xx rate (reliability metric); 404 rate (product metric — indicates abuse/enumeration if spike); 451 rate (abuse takedown metric)
• Latency: TTFB at CDN edge (p50, p99); TTFB at read service (p50, p99, p99.9); S3 GET latency (p99 — this is the tail that controls large-paste p99); Redis GET latency
• Saturation: Redis memory used %; Cassandra pending compactions; S3 throttle errors (503 Slow Down rate); app pod CPU and open connection count

Key alerts (burn-rate rules)

• Page: read availability burn rate > 2% of monthly budget in 1 hour (indicates ~14 minutes of downtime burning in 60 min)
• Page: read p99 TTFB > 200 ms for 5 consecutive minutes (SLO miss territory)
• Page: Cassandra node down (RF=3 means one more failure away from quorum loss)
• Page: S3 error rate > 1% sustained 5 minutes (affects all large-paste reads)
• Ticket: Redis memory > 75% on any shard
• Ticket: any single paste_id > 500 QPS (hot key candidate — engage adaptive caching playbook)
• Ticket: expiry worker queue depth > 24 hours of backlog (worker falling behind)
• Ticket: write latency p99 > 1s sustained (S3 PUT or DB write degradation)

Tracing & debugging

• Trace ID propagated through CDN (via response header) → LB → read service → Redis → Cassandra → S3. A slow paste read can be unwound to the exact tier that added the latency.
• Sampled tracing: 1% baseline; 100% for all 5xx, all requests > 200 ms, and all pastes > 1 MB
• Cache hit/miss as span attributes: cache.l1_hit, cache.redis_hit, cache.s3_fetch — the trace shows exactly which tiers fired
• S3 request ID in trace: every S3 GET/PUT includes AWS request ID as a span attribute; essential for opening S3 support tickets against a specific failed request

xii. Key Takeaways

1. Split metadata from content. Metadata (IDs, expiry, owner, size) belongs in a fast, indexed database. Content bytes belong in object storage. Mixing them — putting 10 MB blobs in Cassandra rows — collapses both tiers simultaneously under load.
2. The inline threshold is a first-class decision. Storing small pastes (<4 KB) inline in the DB row eliminates an entire network hop for the majority of pastes. Tune this threshold based on observed size distribution, DB row limits, and S3 GET latency. It is a config value, not hard code.
3. Write order matters for consistency. S3 PUT before DB write. A failed S3 PUT leaves no trace — the paste simply doesn't exist. A DB write before a failed S3 PUT leaves a dangling metadata row with no content, which is harder to clean up and confusing to read-path code.
4. Expiry is a pipeline, not a flag. Read-time enforcement is the correctness guarantee. The async worker is the storage reclamation mechanism. S3 lifecycle rules are the backstop. All three must exist; each compensates for the others' failure modes.
5. Pre-signed S3 URLs are the bandwidth escape hatch. For large pastes, redirect the client directly to S3. The app tier serves a 307 — not bytes. This keeps app pods stateless and thin, offloads bandwidth entirely to S3's own infrastructure, and is free to implement.
6. Object storage has throughput ceilings per prefix. S3 allows ~3,500 PUT/s and ~5,500 GET/s per prefix. At low QPS this is invisible. At scale, prefix sharding (first 2 chars of the key) multiplies this ceiling by the number of prefixes. Design it in early — retrofitting prefix sharding onto an existing key scheme is painful.
7. Burn-after-read is eventually consistent by design. A race between two simultaneous readers means one gets the content and one gets a 404. Document this, set user expectations, and don't try to make it atomic with distributed locking — the cure is worse than the disease.
8. Abuse is load-bearing, not optional. Pastebin is one of the most abused infrastructure primitives on the internet. The content scanning pipeline, rate limiting, and takedown flow must be built before launch, not after the first incident.
9. The read path is shaped by bytes, not requests. 3,500 QPS sounds modest until you multiply by 10 KB average and realize you need ~280 Mbps sustained at peak — without CDN. Architecture everything around the byte rate, not the request rate. CDN is not optional; it is load-bearing infrastructure.
10. Observability at the content_type boundary. Split all metrics by whether the paste is inline or S3. A latency regression that affects only S3 pastes (large content) looks like a p99 spike but leaves p50 fine — you will miss it without the split. This is the trace attribute that pays for itself on the first incident.

xiii. Go Deeper

• How would you implement real-time collaborative editing (Google Docs–style) on top of this architecture? What changes first — the storage model, the protocol, or the consistency model?
• Design the content abuse pipeline end-to-end: ingestion → async scan → result routing → takedown → appeal. What's your false positive handling? How do you prevent legitimate pastes from being tombstoned?
• Pastebin pastes can embed URLs (phishing). Design a URL scanner that processes pastes without adding latency to the write path. How do you handle pastes with 1,000 embedded links?
• How would you implement a "paste diff" feature — showing the differences between two paste versions — at scale? Where does the diff computation happen, and how do you cache it?
• The expiry worker processes 10M pastes/day at peak. How would you design it to survive a 3-day worker outage and then catch up without overloading Cassandra or S3 on recovery?
• A user claims GDPR right-to-erasure. Their account has 50,000 pastes. Design the deletion pipeline. What's the SLA? What do you do with pastes that have been forked by other users?
• How would you implement paste encryption at rest (client-side, not server-side) so that even the platform cannot read paste content? What does this do to abuse detection?