Harvard's CS 75 Scalability

Resources

• CS75 (Summer 2012) Lecture 9 - Scalability Harvard Web Development - David Malan

Web Hosts

• Some hosts may block IP addresses in certain regions or countries.
• SFTP vs. FTP: SFTP encrypts all traffic (important for credentials). FTP sends usernames and passwords in the clear, which is a critical security risk.
• “Unlimited storage” hosting plans usually oversell resources; you are sharing one machine with hundreds of other users.
• VPS (Virtual Private Server): Still shares hardware, but you get your own OS instance; only you and admins access your files.
• For maximum privacy and control, you may need to run your own servers.
• AWS EC2: Example of an IaaS option offering virtualized servers.

Scaling

Vertical Scaling

• Add more RAM, CPUs, and disks to a single server.
• Ceiling limits:
  • Cost and hardware constraints.
  • State of the art in technology, since no machine has infinite resources.

Horizontal Scaling

• Add multiple (often cheaper) servers.
• Distributes load and avoids the ceiling of a single machine.
• Common in modern system design.

Load Balancing

• Distributes inbound requests across multiple servers.
• Client sees only the load balancer’s public IP.
• Can be implemented by:
  • DNS Round Robin: simple but limited (cache issues, uneven loads).
  • Dynamic load awareness: routing based on server load.
  • Dedicated resources: for example, separate servers for static content (images, video).
• Load Balancing Tech:
  • Software: ELB (AWS), HAProxy, LVS.
  • Hardware: Barracuda, Cisco, Citrix, F5 (expensive).

Shared Session State

• With multiple servers, sessions stored locally can be lost if requests hit a different server.
• Solutions:
  • Centralized session storage (file server, database, or NFS).
  • Sticky sessions (session affinity).

Session Affinity (Sticky Sessions)

• Ensures requests from the same user hit the same server.
• Approaches:
  • Cookies with server IDs: brittle, because if a server dies the cookie may keep sending the user to a dead server.
  • Load balancer-managed mapping: better, because the load balancer assigns a random ID and handles the logic.

Storage and RAID

• RAID applies to disks within a single server, not replication across servers.
• RAID improves redundancy and performance:
  • RAID 0: Striping (fast, no redundancy).
  • RAID 1: Mirroring (redundancy, doubles space needed).
  • RAID 5/6: Parity-based redundancy.
  • RAID 10: Striping plus mirroring.
• Reduces downtime risk from disk failures.
• Shared storage tech:
  • Fiber Channel, iSCSI for high-speed SAN.
  • NFS for shared filesystems.
  • Databases (MySQL) for session storage.

Database Replication

Primary-Replica (Active:Passive)

• Primary handles writes, replicas keep synchronized copies.
• Replicas improve read scalability.
• Failover is possible but involves downtime.

Primary-Primary (Active:Active)

• Writes allowed on multiple primaries.
• Provides higher availability and redundancy.
• Complexity comes from conflict resolution.

Load Balancing plus Replication

• Can use active-passive load balancers or active-active pairs.
• Passive balancer can auto-promote itself if the active one fails.

Partitioning (Sharding)

• Split data across servers based on rules (for example, A-M vs. N-Z users).
• Enables horizontal database scaling.
• Catch: cross-partition operations (for example, user pokes someone on a different server) become more complex.

Caching

Types

• Static HTML vs. dynamic DB-driven pages: Static is fast but harder to update.
• MySQL Query Cache: Stores results of identical SQL queries, but must be invalidated when data changes.
• Memcached: Distributed in-memory key-value store.
• PHP Acceleration: Keeps compiled opcodes in memory, avoiding re-parsing scripts each request.

Benefits

• Reduces database load.
• Speeds up repeated queries or frequently accessed content.

Data Center Redundancy

• Multiple geographically distributed data centers mitigate outages and disasters.
• DNS directs traffic to the nearest or healthiest center.
• AWS Availability Zones: physically separate buildings or clusters within a region, isolated from each other’s failures.

Security

• Firewalls restrict unnecessary ports (least privilege principle).
• HTTP and HTTPS for web traffic; database ports only for DB communication.
• Helps contain breaches and reduce attack surface.

Progression

Step 1: Single Server

┌───────────────┐
│   Web + DB    │
└───────────────┘

Step 2: Vertical Scaling

┌───────────────┐
│  Bigger Box   │
│ (More CPU/RAM)│
└───────────────┘

Step 3: Horizontal Scaling + Load Balancing

           ┌─────────────┐
           │Load Balancer│
           └───────┬─────┘
                   │
      ┌────────────┴────────────┐
      │                         │
┌───────────────┐        ┌───────────────┐
│   Web + DB    │  ...   │   Web + DB    │
└───────────────┘        └───────────────┘

Step 4: Shared Sessions and Caching

           ┌─────────────┐
           │Load Balancer│
           └───────┬─────┘
                   │
      ┌────────────┴───────────┐
      │                        │
┌─────────────┐        ┌─────────────┐
│ Web Servers │  ...   │ Web Servers │
└─────┬───────┘        └───────┬─────┘
      │ Shared Session + Cache │
      └───────────┬────────────┘
                  ▼
          ┌───────────────┐
          │ Cache (Memory)│
          └───────────────┘

Step 5: Database Replication

           ┌───────────────┐
           │    Primary    │
           └───────┬───────┘
                   │ Replication
        ┌──────────┴───────────┐
        │                      │
┌───────────────┐       ┌───────────────┐
│  Replica DB   │  ...  │  Replica DB   │
└───────────────┘       └───────────────┘

Step 6: Partitioning (Sharding)

Users A-M  ──► DB Shard 1
Users N-Z  ──► DB Shard 2

Step 7: Data Center Redundancy

   ┌───────────────┐        ┌───────────────┐
   │ Data Center 1 │        │ Data Center 2 │
   └───────────────┘        └───────────────┘
          │                         │
          └──────────DNS────────────┘

Summary

             ┌────────────────────┐
             │   Internet         │
             │ (Clients/Browsers) │
             └─────────┬──────────┘
                       │  TCP 80/443
             ┌─────────▼──────────┐
             │   Load Balancers   │
             │ (Active/Passive or │
             │   Active/Active)   │
             └───────┬─┬──────────┘
                     │ │
        ┌────────────┘ └─────────────┐
        │                            │
┌───────▼─────────┐        ┌─────────▼───────┐
│   Web Servers   │        │   Web Servers   │
│ (App / PHP/etc) │  ...   │ (App / PHP/etc) │
└───────┬─────────┘        └─────────┬───────┘
        │  (Shared Session / Cache)  │
        │          │                 │
        └──────────┴─────────────────┘
                   │
          ┌────────▼─────────┐
          │   In-Memory      │
          │   Caching Layer  │
          │ (Memcached/Redis)│
          └────────┬─────────┘
                   │
         ┌─────────▼─────────┐
         │  Database Tier    │
         │                   │
         │   ┌───────────┐   │
         │   │  Primary  │◄──┐
         │   └─────┬─────┘   │
         │         │ Replication
         │   ┌─────▼─────┐   │
         │   │ Replicas  │   │
         │   └───────────┘   │
         └───────────────────┘
                   │
          ┌────────▼─────────┐
          │   Storage Layer  │
          │  (RAID / SAN /   │
          │   NFS / iSCSI)   │
          └──────────────────┘

          [ Firewalls between tiers ]