WebSockets, Schedulers, Notifications & Real-Time Pipelines — Interview Questions

How to use this lesson

Real-time features and background jobs fail in production for the same reasons:

• state held on one node;
• duplicate execution across instances;
• missing retries or back-pressure.

Design them stateless, idempotent and horizontally scalable, with a durable queue or store behind them.

Q1. How does Spring handle WebSocket communication?

Short answer:

• The raw WebSocket API: implement WebSocketHandler (TextWebSocketHandler), and register it with @EnableWebSocket + WebSocketConfigurer. It's low-level (you manage the sessions and the message format).
• STOMP over WebSocket (common in Spring MVC): @EnableWebSocketMessageBroker:
  • clients connect (with SockJS fallback, optional), and subscribe to destinations (/topic/prices, /user/queue/notifications);
  • @MessageMapping("/chat.send") controller methods handle the inbound messages;
  • SimpMessagingTemplate.convertAndSend(...) pushes from anywhere.
• The broker: an in-memory simple broker (single node only), or a broker relay to RabbitMQ or ActiveMQ (STOMP plugin), which enables multi-instance fan-out.
• WebFlux: a reactive WebSocketHandler (Flux<WebSocketMessage>), or RSocket.
• Alternatives: Server-Sent Events (one-way server → client over HTTP: simpler, auto-reconnect, proxy-friendly), for dashboards and notifications.

@Configuration @EnableWebSocketMessageBroker
class WsConfig implements WebSocketMessageBrokerConfigurer {
    public void registerStompEndpoints(StompEndpointRegistry r) { r.addEndpoint("/ws").setAllowedOrigins("https://app.shop.example"); }
    public void configureMessageBroker(MessageBrokerRegistry r) {
        r.setApplicationDestinationPrefixes("/app");
        r.enableStompBrokerRelay("/topic", "/queue").setRelayHost("rabbitmq").setRelayPort(61613);   // multi-node fan-out
        r.setUserDestinationPrefix("/user");
    }
}

Q2. How do you secure WebSocket endpoints?

Short answer:

• Authenticate the handshake: the HTTP upgrade request goes through the Spring Security filter chain (session cookie or bearer token). Browsers can't set custom headers on WebSocket connections, so use cookies (with CSRF or origin protections), a short-lived ticket token in the query string (exchanged over authenticated REST, with a single use), or the STOMP CONNECT frame headers, validated by a ChannelInterceptor.
• Authorise messages: Spring Security's messaging support (@EnableWebSocketSecurity, AuthorizationManager<Message<?>> rules per destination: only the owner can subscribe to /user/queue/..., and admins to /topic/admin/**).
• Origin checks: setAllowedOrigins (prevents cross-site WebSocket hijacking).
• Transport: wss:// only.
• Abuse limits: message size limits, rate limiting per session, idle timeouts, a maximum number of connections per user, and input validation on every message.
• Token expiry: re-authenticate or close long-lived connections when the token expires.

Q3. How would you design a chat system with WebSockets?

Short answer:

• Connections: clients keep WebSocket (STOMP) connections to stateless gateway nodes behind a load balancer (sticky sessions are optional with a broker relay).
• Message flow:
  1. the client sends a message → the gateway authenticates and authorises it (conversation membership) →
  2. the message is persisted (Cassandra or ScyllaDB, partitioned by conversation ID and time bucket, or Postgres for smaller scale), with a server-generated ID and timestamp →
  3. it's published to the conversation's topic (Kafka, or Redis pub/sub) →
  4. the gateway nodes holding the recipients' connections push it (presence registry: user → node, in Redis) →
  5. offline users get it through push notifications (FCM or APNs), and fetch history on reconnect.
• Features: delivery and read receipts (acknowledgements), typing indicators (ephemeral, not persisted), ordering per conversation (a Kafka partition key, or sequence numbers), idempotent client message IDs (deduplicate resends), history pagination, media via object storage with pre-signed URLs, and end-to-end encryption if required.
• Scale: horizontal gateways (each handles tens of thousands of connections with NIO or virtual threads), connection draining on deployment, and back-pressure for slow consumers.

Learn it in depth → Design WhatsApp

Q4. How would you design a real-time dashboard system? How do you scale a WebSocket-based dashboard?

Short answer:

• The data pipeline:
  1. events or metrics → Kafka;
  2. → stream processing (Kafka Streams, Flink) computing windowed aggregates (orders per minute per region, revenue);
  3. → a fast serving store (Redis for the current values, and ClickHouse, Druid or Pinot for time-series queries).
• The push layer: a stateless push service subscribes to the aggregate updates (a Kafka topic, or Redis pub/sub), and pushes deltas to subscribed clients over SSE or WebSockets. Clients load their initial state through REST, then apply the incremental updates.
• Scaling the WebSocket tier:
  • stateless nodes, with fan-out through a pub/sub backplane (Redis, Kafka, or a STOMP broker relay), so any node can deliver to its connected clients;
  • an L4/L7 load balancer that supports WebSockets, with long idle timeouts;
  • throttle and coalesce updates (send at most every 500 ms per client, instead of per event);
  • send deltas, and compress them;
  • per-connection back-pressure (drop stale intermediate updates for slow clients);
  • autoscale on the connection count;
  • graceful draining during deployments (clients reconnect with backoff and jitter, to avoid thundering herds).
• Security: authorisation per dashboard or tenant on subscription.

Learn it in depth → Design a Real-Time Analytics Dashboard

Q5. How do you schedule recurring tasks in Spring Boot?

Short answer:

• @EnableScheduling + @Scheduled: fixedRate/fixedDelay/initialDelay (with duration strings like "PT5M"), and cron = "0 0 2 * * *" (6 fields, seconds included) with a zone. Externalise the schedules (cron = "${jobs.cleanup.cron}").
• Configure the scheduler pool: spring.task.scheduling.pool.size (the default is 1 thread, so one slow job delays the others), or virtual threads.
• SchedulingConfigurer for dynamic schedules (cron expressions from the database).
• For robust jobs: Quartz (persistent, clustered), Spring Batch (chunked batch processing, with restartability) launched by a scheduler, or an external scheduler (a Kubernetes CronJob, cloud schedulers).
• Always: make the jobs idempotent, cluster-safe (Q8), observable (metrics, last success time), and bounded in duration.

Q6. What's the difference between Quartz and Spring's @Scheduled?

Short answer:

Other options: db-scheduler (a lightweight, persistent, cluster-safe library), JobRunr, and platform CronJobs.

Q7. How do you implement distributed cron jobs? How do you make a scheduler cluster-safe (no double firing)?

Short answer:

• Locking with @Scheduled: use ShedLock (@SchedulerLock(name = "dailyReport", lockAtMostFor = "PT30M", lockAtLeastFor = "PT1M")), with a lock provider (a JDBC table, Redis, Mongo or ZooKeeper). Only the instance that acquires the lock runs the job. lockAtMostFor protects against crashed holders, and lockAtLeastFor prevents re-runs caused by clock skew. Caveat: it doesn't guarantee exactly-once under long GC pauses, so the job must still be idempotent.
• Quartz clustered mode: triggers are coordinated through the database (org.quartz.jobStore.isClustered=true).
• Leader election: a Kubernetes Lease, Spring Integration LockRegistryLeaderInitiator, ZooKeeper or etcd. Only the leader runs the schedules.
• Externalise the scheduling:
  • a Kubernetes CronJob (concurrencyPolicy: Forbid, startingDeadlineSeconds), running a one-off container;
  • cloud schedulers (EventBridge Scheduler, Cloud Scheduler) that enqueue a message, consumed by exactly one worker from a queue;
  • workflow engines (Temporal schedules) for durable, observable jobs.
• Idempotency and resumability: track the run keys (job + scheduled time) in a table, so duplicates are ignored. Process in resumable chunks.

@Scheduled(cron = "${jobs.expire-holds.cron:0 */5 * * * *}", zone = "Asia/Kolkata")
@SchedulerLock(name = "expireBookingHolds", lockAtMostFor = "PT4M", lockAtLeastFor = "PT30S")
public void expireHolds() { holds.expireOlderThan(clock.instant().minus(Duration.ofMinutes(15))); }   // idempotent

Q8. How do you secure scheduled background jobs?

Short answer:

• Identity and least privilege: jobs run with a dedicated service identity (a technical user, or a service account), whose permissions cover exactly what the job does. With method security, run the job inside a system security context (SecurityContextHolder with a service principal), or through client-credentials tokens when calling other services.
• Secrets: from the secret manager (no hard-coded credentials in the job configuration).
• Triggering: if jobs can be triggered manually (admin endpoints, Actuator, a Quartz UI), protect those endpoints (admin roles, internal network only), and audit every manual run.
• Data safety: guard destructive jobs (dry-run modes, limits per run, safety checks against deleting too much), and handle tenant isolation in multi-tenant jobs.
• Observability and audit: log each run with its correlation ID, outcomes and counts. Alert on failures, and on "didn't run" (a missing heartbeat, using a dead man's switch like Healthchecks.io).
• Supply chain: Kubernetes CronJob images scanned, running as non-root, with a restricted service account and network policies.

Q9. How would you design a cron-based report scheduler?

Short answer:

• Model: report_schedule(id, tenant_id, report_type, params, cron, timezone, recipients, format, next_run_at, enabled). Users define the schedules in their own time zones (store the zone, and compute next_run_at in UTC, handling DST with CronExpression/ZonedDateTime).
• Triggering:
  • a dispatcher (cluster-safe: ShedLock, Quartz, db-scheduler, or a SELECT … FOR UPDATE SKIP LOCKED WHERE next_run_at <= now() poller) finds the due schedules;
  • it enqueues report jobs (Kafka or SQS) with an idempotency key (schedule_id, scheduled_time);
  • it advances next_run_at in the same transaction.
• Execution: workers (autoscaled on queue depth) generate reports asynchronously (streaming queries against read replicas or the warehouse, never the OLTP primary during peaks), write the output to object storage, then notify (an email with a pre-signed link, not a huge attachment; or in-app).
• Robustness: retries with backoff, a DLQ, timeouts, per-tenant concurrency limits (so one tenant can't hog the workers), catch-up policies for missed runs (skip or run once), and run history with status.
• Security: reports respect the data permissions of the schedule owner at run time (the user may have lost access since), and links expire.

Q10. How do you implement email scheduling and retries?

Short answer:

• Decouple sending from business transactions: write an email outbox or notification record (to, template, data, send_at, status, idempotency key) in the business transaction, or publish an event.
• A scheduled dispatcher picks due messages (send_at <= now, cluster-safe with SKIP LOCKED), and sends them through the provider API (SES, SendGrid) with the idempotency key.
• Retries: exponential backoff with jitter for transient errors (429, 5xx, timeouts); no retries for permanent errors (invalid address, a hard bounce). There's a maximum number of attempts, then dead-letter status with an alert. RabbitMQ delayed queues or Kafka retry topics are alternatives.
• Provider feedback: process bounce and complaint webhooks, and suppress bad addresses (deliverability and reputation).
• Scheduling features: time zones (send at 9 AM local time), quiet hours, rate limits per provider, batching, and user preferences and unsubscribes (legal compliance).
• Observability: delivery metrics, and a latency SLO from event to sent.

Q11. How do you design a scalable notification system?

Short answer:

• Ingestion: services publish notification requests or events (Kafka), or call a Notification API (idempotency keys).
• Processing:
  • template rendering and localisation;
  • user preferences (channels, opt-outs, quiet hours, frequency caps);
  • deduplication and aggregation ("5 new comments");
  • priority lanes (OTPs and security alerts are separate from marketing).
• Channel workers: per channel (email, SMS, push, in-app, WhatsApp), each with its own queue, scaling and rate limits, and provider failover (a secondary SMS vendor).
• Delivery tracking: status (queued, sent, delivered, failed) from provider callbacks, retries with backoff, DLQs, and an in-app inbox stored per user.
• Scale: partition by user ID (ordering, and aggregation per user), autoscale the workers on queue lag, cache the preferences, and batch the provider calls.
• Compliance and safety: unsubscribe links, consent records, personal data minimisation, and security-sensitive notifications bypassing marketing caps.

Learn it in depth → Design a Notification Service

Q12. How would you design a video transcoding pipeline?

Short answer:

1. Upload: clients upload directly to object storage (pre-signed multipart or resumable uploads). A metadata record (PENDING) is created.
2. Trigger: a storage event (S3 → SQS or EventBridge) or a Kafka message starts a job.
3. Orchestration: a workflow (Step Functions, Temporal, or a custom state machine) validates the input (probe with ffprobe: format, duration, codecs; plus a malware scan), then fans out transcoding tasks per rendition (1080p, 720p, 480p; H.264, HEVC or AV1), and segments for adaptive streaming (HLS/DASH). Long videos are split into chunks, transcoded in parallel, then stitched.
4. Workers: stateless containers running FFmpeg (GPU-accelerated where it's cost-effective), pulling tasks from a queue, and autoscaled on queue depth (spot instances for cost). Or use managed services (AWS Elemental MediaConvert).
5. Output: renditions, manifests, thumbnails and captions written to object storage, and served through a CDN with signed URLs or DRM if required.
6. Reliability: idempotent tasks (keyed by video, rendition and chunk), retries and DLQs, progress tracking, timeouts, and a final status update plus notification (READY).
7. Observability: per-stage metrics, failure rates by input type, cost per minute transcoded.

Follow-up questions this topic invites — and their answers

Q: SSE or WebSockets for a live dashboard? A: SSE when updates flow only from server to client: it's plain HTTP (it works through proxies, with auto-reconnect and Last-Event-ID resume), and it's simpler to scale. Use WebSockets when you need bidirectional, low-latency messaging (chat, collaborative editing, games).

Q: How do you handle time zones and DST in cron schedules? A: Store the user's time zone, and evaluate the cron in that zone (Spring's @Scheduled(zone = …), or CronExpression.next(ZonedDateTime)). Define the behaviour for skipped or repeated local times during DST transitions, and store the run instants in UTC.

Q: What is SELECT … FOR UPDATE SKIP LOCKED good for? A: Building a database-backed work queue. Several workers grab different due rows concurrently, without blocking each other or processing the same row twice (Postgres, MySQL 8, Oracle).

Q: How many WebSocket connections can one Java node hold? A: With NIO (Netty or Tomcat NIO) or virtual threads, tens of thousands to hundreds of thousands, limited by memory per connection, file descriptors (the ulimit), CPU for message fan-out, and the load balancer's limits. Test with realistic message rates.