Step 1 - Challenge: Protocol Handling

Step 1 - Challenge: Protocol Handling

In this step your goal is to build the functionality to serialise and de-serialise Redis Serialisation Protocol (RESP) messages. This is the protocol used to communicate with a Redis Server. You may want to refer to the RESP protocol specification.

Redis uses RESP as a request-response protocol in the following way:

  • Clients send commands to a Redis Server as a RESP Array of Bulk Strings.

  • The server replies with one of the RESP types according to the command implementation.

In RESP, the first byte determines the data type:

  • For Simple Strings, the first byte of the reply is "+"

  • For Errors, the first byte of the reply is "-"

  • For Integers, the first byte of the reply is ":"

  • For Bulk Strings, the first byte of the reply is "$"

  • For Arrays, the first byte of the reply is "*"

RESP can represent a Null value using a special variation of Bulk Strings: "$-1\r\n" or Array: "*-1\r\n".

Your challenge now, is to write the code required to serialise and de-serialise messages. I suggest you build on the tests in the previous lesson and build tests for some example messages, i.e.:

  • "$-1\r\n"

  • "*1\r\n$4\r\nping\r\n”

  • "*2\r\n$4\r\necho\r\n$5\r\nhello world\r\n”

  • "*2\r\n$3\r\nget\r\n$3\r\nkey\r\n”

  • "+OK\r\n"

  • "-Error message\r\n"

  • "$0\r\n\r\n"

  • "+hello world\r\n”

Plus some invalid test cases to test outside the happy path. It makes sense to work through them in the order of the types listed above, i.e. string, then errors then integers and so on.

You should create the following directory structure for your project:

ccpyredis
  README.md
  pyredis
    protocol.py
  tests
    test_protocol.py
  venv

Update your project README with details of how to run the tests to verify the correct functionality of the protocol handling code.

Build Your Own Redis Server (Python Edition)

Buy nowLearn more

Introduction

  • Introduction
  • Learn About Redis

Step 0 - Setting Up

  • Step Goals
  • Installing Redis
  • Setting Up Python
  • Learn To Use Redis, Redis CLI And Redis Benchmark
  • Create A GitHub Repo For Your Project.1

Step 1 - Introduction to RESP, Building a Protocol Handler Using Test-Driven Development

  • Introduction to RESP
  • Introduction to Test Driven Development
  • Using Test Driven Development to Build a Protocol Handler
  • Step 1 - Challenge: Protocol Handling
  • Step 1 - Solution: Protocol Handling5

Step 2 - Creating a Network Client and Server

  • Introduction To Creating A Network Client And Server4
  • Serialising the RESP Data Types2
  • Building a Simple Redis CLI
  • Testing the CLI Against Redis
  • Sample Solution for a Simple Redis CLI8
  • Step 2 - Challenge: Handling PING and ECHO Commands
  • Building a Simple Server To Handle Incoming Connections
  • Responding To The PING And ECHO Commands
  • Step 2 - Solution: Handling PING and ECHO Commands3

Step 3 - Data Structures, Set and Get

  • Set Up Continuous Integration
  • Step 3 - Challenge: SET and GET
  • Adding The Core Data Structure
  • Handling The Set Command
  • Handling the Get command
  • Using the Redis CLI to test Set and Get
  • Step 3 - Solution: SET and GET4

Step 4 - Handling Concurrency

  • Step 4 - Challenge: Handling Concurrent Clients1
  • Threads Versus Async
  • Making The Data Store Thread-Safe
  • Handling Concurrent Clients With Threads
  • Using The Redis CLI And Redis Benchmark To Test Concurrency
  • Step 4 - Solution: Handling Concurrent Clients With Threads
  • Step 4 - Async Solution: Handling Concurrent Clients With Async IO2

Step 5 - Implementing Expiry

  • Introducing The Two Methods Of Expiry
  • Step 5 - Challenge: Implementing Key Expiry
  • Updating The Data Store And Set Command Handler Ready For Expiry
  • Implementing Expire On Read2
  • Implementing Background Expiry
  • Step 5 - Solution: Implementing Key Expiry (Common Parts)1
  • Step 5 - Solution: Implementing Key Expiry Using Threads
  • Step 5 - Solution: Implementing Key Expiry Asynchronously

Step 6 - Implementing More Commands

  • Step 6 - Challenge: Adding More Commands2
  • Impelenting EXISTS and DEL
  • Implementing INCR and DECR3
  • Implementing LPUSH, RPUSH And LRANGE2
  • Step 6 - Solution: Adding More Commands2

Step 7 - Performance

  • Step 7 - Challenge: Performance Optimisation and Benchmarking
  • Benchmarking Your Server
  • How To Optimise For Performance
  • Performance Tuning The Server3

Step 8 - Persistence

  • Step 8 - Challenge: Persistence
  • Persisting The Database
  • Loading The Database
  • Step 8 - Solution: Persistence

Closing Remarks

  • The Complete Final Solution
  • Going Further
  • Wrapping Up