# Hooks Rewatch Signal Service

## Overview

A real-time stream processing service built on Apache Flink that tracks user engagement signals related to hook rewatching behavior. The service processes rewatch events from hooks (short video clips) and stores engagement signals in Redis for use by recommendation systems to understand user preferences and engagement patterns.

## Event Schema

The service processes `HookRewatch` events with the following structure:

```json
{
  "name": "HookRewatch",
  "source": "mobile",
  "user_id": "123",
  "timestamp": "{ISO-timestamp}",
  "properties": {
    "hook_id": "abc-def-123-456"
  }
}
```

## Data Storage

- **Redis Key**: `hooks_positive_signal_rewatch:{user_id}`
- **Data Structure**: Sorted set with hook IDs as members and timestamps as scores
- **Retention**: Only the most recent 200 rewatch events per user are kept
- **Purpose**: Used by recommendation algorithms to identify user preferences based on rewatching behavior

## Architecture

### Components
- **Kinesis Stream**: Consumes events from `rec-events-stream`
- **Flink Application**: Processes events in real-time
- **Redis**: Stores user engagement signals
- **CloudWatch**: Logging and monitoring

### Processing Flow
1. Consumes `HookRewatch` events from Kinesis
2. Validates event structure (requires `user_id` and `hook_id`)
3. Stores engagement signal in Redis sorted set
4. Maintains sliding window of 200 most recent events per user
5. Logs processing results for monitoring

## Configuration

### Environment Variables
- `stream.arn`: Kinesis stream ARN
- `redis.host`: Redis cluster endpoint
- `redis.port`: Redis port (default: 6379)
- `env.stage`: Environment stage (dev/staging/prod)

### Flink Configuration
- **Parallelism**: 4
- **Checkpointing**: Enabled (1-minute intervals)
- **Processing Mode**: Exactly-once
- **State TTL**: 36 hours

### Kinesis Reader Configuration
- **Development**: POLLING mode (simpler setup, no consumer management)
- **Production/Staging**: Enhanced Fan-Out (EFO) with dedicated consumer for better performance
- **EFO Consumer**: `hooks-rewatch-signal-consumer-{env}`
- **EFO Lifecycle**: Self-managed

## Deployment

The service is deployed as a Kinesis Data Analytics application using AWS CDK:

- **Application Name**: `hooks-rewatch-signal-app-{stage}`
- **Consumer Name**: `hooks-rewatch-signal-consumer-{stage}`
- **Log Group**: `/aws/kinesis-analytics/hooks-rewatch-signal-app-{stage}`

## Monitoring

### CloudWatch Logs
- Application logs are sent to CloudWatch
- Processing results logged with identifier: `HOOK-REWATCH-EVENTS`

### Metrics
- Flink application metrics available in CloudWatch
- Custom metrics for processing success/failure rates

## Local Development

For local development, ensure you have:
1. Python environment with PyFlink dependencies
2. Local Redis instance running on port 6379
3. Kinesis connector JAR file in the `jar/` directory

## Related Services

This service is part of the recommendation infrastructure alongside:
- `hooks-omniplay-signal`: Tracks omni player opening engagement
- `hooks-comment-view-signal`: Tracks comment viewing engagement
- `hooks-profile-view-signal`: Tracks profile viewing engagement
- Other signal processing services for collaborative filtering

## Signal Significance

Rewatch events are considered strong positive signals for recommendation algorithms because:
- They indicate genuine user interest beyond initial viewing
- Users who rewatch hooks are more likely to be engaged with similar content
- Rewatch behavior is a strong predictor of user preferences
- The temporal nature (timestamp scoring) allows for recency-based recommendations