Redis Data Structures and Key Patterns

Field	Value
Type	Skill Resource
Source	~/.copilot/skills/database/references/redis.md
Description	Not specified

Source Content

Redis Data Structures and Key Patterns

This is a lighter reference for Redis fundamentals; Postgres is the primary engine in this skill. Expand this reference as caching and real-time workloads appear.

Data Structures

Redis is a key-value cache and data structure server. Choose the right structure for your access pattern.

Strings

Simplest: atomic values, counters, session data.

SET key value [EX seconds] [NX|XX]
GET key
INCR counter
DECR counter
APPEND key " more"
STRLEN key

Example: Session storage

SET session:abc123def "user_id=42&name=Alice&cart=[]" EX 3600
GET session:abc123def

Hashes

Object-like: store multiple fields per key.

HSET user:123 name Alice email alice@example.com age 30
HGET user:123 name
HGETALL user:123
HINCRBY user:123 age 1
HDEL user:123 age

Better than strings for structured data; reduces key explosion.

Lists

Ordered sequences: queues, logs, leaderboards.

LPUSH mylist value1 value2 value3  -- Push to head
RPUSH mylist value4                -- Push to tail
LLEN mylist
LRANGE mylist 0 -1                 -- All elements
LPOP mylist                         -- Pop from head
RPOP mylist                         -- Pop from tail
LTRIM mylist 0 99                  -- Keep first 100 elements

Example: Job queue

LPUSH jobs:pending '{"task": "send_email", "to": "alice@example.com"}'
RPOP jobs:pending  -- Consumer pulls jobs from tail

Sets

Unordered, unique values: tags, memberships, relationships.

SADD myset member1 member2 member3
SMEMBERS myset
SCARD myset
SISMEMBER myset member1
SREM myset member1
SINTER set1 set2                   -- Intersection
SUNION set1 set2                   -- Union
SDIFF set1 set2                    -- Difference

Example: User followers

SADD followers:alice bob charlie dave
SADD followers:bob alice charlie
SINTER followers:alice followers:bob  -- Mutual followers

Sorted Sets

Ordered by score: leaderboards, time-series, priority queues.

ZADD leaderboard 100 alice 95 bob 85 charlie
ZRANGE leaderboard 0 -1            -- By score ascending
ZREVRANGE leaderboard 0 -1         -- By score descending
ZRANK leaderboard alice            -- Position ascending
ZREVRANK leaderboard alice         -- Position descending
ZSCORE leaderboard alice
ZINCRBY leaderboard 10 alice       -- Increment score
ZREM leaderboard alice
ZCOUNT leaderboard 80 100          -- Count in score range

Example: Leaderboard

ZADD game:scores 1500 player_1 1200 player_2 950 player_3
ZREVRANGE game:scores 0 9 WITHSCORES  -- Top 10 with scores

Streams (Redis 5.0+)

Time-series event log: messages, metrics, activity tracking.

XADD mystream * field1 value1 field2 value2
XLEN mystream
XRANGE mystream - +                -- All messages
XREAD COUNT 10 STREAMS mystream 0  -- Read first 10
XDEL mystream message_id

Example: Activity log

XADD user:123:activity * action "login" ip "192.168.1.1" timestamp "2024-01-15T10:00:00Z"
XRANGE user:123:activity - + LIMIT 10  -- Recent 10 activities

Key Naming Patterns

Use colons to organize related keys:

-- User data
user:123:name Alice
user:123:email alice@example.com
-- Or hash:
HSET user:123 name Alice email alice@example.com

-- Session
session:abc123def

-- Cache
cache:product:456
cache:user:789:profile

-- Counters
counter:api:requests:today
counter:page:visits:article_1

-- Queues
queue:email:pending
queue:notifications:retry

-- Sets (relationships)
followers:alice
following:alice

-- Leaderboards
leaderboard:weekly:scores
leaderboard:alltime:scores

-- Sorted sets (time-series)
metrics:api:latency:2024-01-15
metrics:api:errors:2024-01-15

Expiration (Time-To-Live)

Every key can auto-expire. Useful for sessions, caches, temporary data.

SET temp_key value EX 3600         -- Expire in 3600 seconds (1 hour)
SET temp_key value PX 3600000      -- Expire in 3600000 milliseconds
EXPIRE key 60                      -- Set expiration on existing key
TTL key                            -- Check remaining seconds (-1 = no expiry, -2 = expired)
PERSIST key                        -- Remove expiration

Example: Session with auto-expiry

SET session:user123 '{"user_id":123,"role":"admin"}' EX 1800  -- 30 min

Pipeline and Transactions

Pipeline (Batch Commands)

Send multiple commands at once for efficiency:

MULTI
INCR counter:api:requests
ZADD leaderboard 1 player_1
HSET user:123 last_action "api_call"
EXEC

Transactions (WATCH for Optimistic Locking)

WATCH mykey
-- Check value
GET mykey
-- If unchanged, execute
MULTI
SET mykey newvalue
INCR counter
EXEC  -- Succeeds only if mykey wasn't modified

Eviction Policies

When Redis hits maxmemory, it evicts keys based on the policy:

CONFIG SET maxmemory-policy allkeys-lru  -- Evict least recently used key
CONFIG SET maxmemory-policy volatile-lru  -- Evict LRU key with TTL
CONFIG SET maxmemory-policy volatile-ttl  -- Evict key with shortest TTL

Policies:

• noeviction — Error if full (default).
• allkeys-lru — Evict any key, LRU order.
• allkeys-lfu — Evict any key, least frequently used.
• volatile-lru — Evict expiring keys, LRU order.
• volatile-lfu — Evict expiring keys, LFU order.
• volatile-ttl — Evict expiring keys with shortest TTL.
• random — Evict any random key.

Recommendation: Use volatile-lru or allkeys-lfu for caches; noeviction for critical data backed by persistence.

Persistence

Redis is in-memory; enable persistence to survive restarts.

RDB (Snapshots)

Periodic full snapshots. Fast recovery, simple, but can lose recent writes.

SAVE  -- Blocking snapshot
BGSAVE  -- Non-blocking background snapshot

AOF (Append-Only File)

Write-ahead log; more durable but slower writes.

CONFIG SET appendonly yes  -- Enable AOF
CONFIG SET appendfsync everysec  -- Sync every second (good balance)

Recommendation: Use BGSAVE for backups; enable AOF on production for durability.

Pub/Sub (Simple Messaging)

Publish messages to channels; subscribers receive immediately (no persistence).

-- Subscriber
SUBSCRIBE channel:notifications
-- (receives messages in real-time)

-- Publisher
PUBLISH channel:notifications '{"type":"order_shipped","order_id":123}'

Not suitable for reliable queues. Use Streams for persistent event logs.

When to Use Redis

Good for:

• Session storage (user auth, cart, preferences).
• Caching (database query results, API responses).
• Rate limiting (counter per IP, per user).
• Real-time analytics (counters, leaderboards).
• Job queues and task processing.
• Real-time pub/sub messaging.
• Temporary data (OTPs, password reset tokens).

Don’t use Redis for:

• Primary data storage (use Postgres/MySQL).
• Durable event logs (use Kafka or Streams with AOF).
• Complex queries or JOINs.
• Very large datasets (memory is expensive).

Common Patterns

Rate Limiting (Sliding Window)

INCR rate_limit:ip:192.168.1.1
EXPIRE rate_limit:ip:192.168.1.1 60  -- 1-second window
-- Check: if value > limit, reject

Distributed Lock

SET lock:resource "unique_token" NX EX 10
-- If successful, you have the lock for 10 seconds
-- Do work
DEL lock:resource  -- Release

Leaderboard with Time Decay

-- Every game, increment by score
ZADD leaderboard:weekly <score> <player>
-- Every week, archive and reset
RENAME leaderboard:weekly leaderboard:weekly:old
-- New week starts fresh

Performance Tips

1. Use pipelining for batch operations.
2. Set sensible TTLs to avoid memory bloat.
3. Monitor memory usage: INFO memory.
4. Use hashes instead of multiple strings for related data.
5. Prefer sorted sets over lists for ordered data you query.
6. Keep individual values < 1 MB.
7. Use AOF with everysec for persistence without killing throughput.

References

• Redis Documentation
• Redis Data Types
• Redis Commands
• Redis Patterns — common use cases