markmywords/README.md

# markMyWords

A self-hostable Docker application that automatically pulls markdown repositories at user-defined intervals and displays them as searchable HTML pages.

## Features

- 🚀 **Self-hosted**: Run entirely on your own infrastructure
- 📦 **Docker-ready**: Simple Docker Compose setup
- 📅 **Scheduled Pulls**: Configure cron-style schedules for each repository
- 🔍 **Full-text Search**: Search across all markdown files
- 📄 **Markdown Rendering**: Beautiful HTML rendering with syntax highlighting
- ⚡ **Zero Configuration**: Works out of the box with configuration file

## Quick Start

### 1. Clone and Configure

```bash
cd markmywords
# Create the config directory
mkdir -p config

# Copy the example configuration
cp config/repositories.json.example config/repositories.json

# Edit the configuration with your repositories
nano config/repositories.json
```

### 2. Configure Repositories

Edit `config/repositories.json` to specify which repositories to pull:

```json
{
  "repo-name": {
    "url": "https://github.com/username/repo.git",
    "enabled": true,
    "schedule": "0 */6 * * *"
  }
}
```

**Schedule Format**: Standard cron expression (minute, hour, day of month, month, day of week)

Common schedules:
- `0 * * * *` - Every hour
- `0 */6 * * *` - Every 6 hours (default)
- `0 9 * * *` - Daily at 9 AM
- `0 0 * * 0` - Weekly (Sunday at midnight)

### 3. Run with Docker Compose

```bash
docker-compose up -d
```

The application will be available at `http://localhost:5000`

### 4. Access the Application

- **Main Page**: http://localhost:5000
  - View all repositories and their markdown files
  - Search across all files

- **View File**: Click on any markdown file to view it as HTML

- **API Endpoints**:
  - `/api/status` - Application status and repository info
  - `/api/search?q=<query>` - Search markdown files

## Configuration

### repositories.json

```json
{
  "repo-name": {
    "url": "https://github.com/username/repo.git",
    "enabled": true,
    "schedule": "0 */6 * * *"
  }
}
```

**Fields**:
- `url`: Git repository URL (HTTPS recommended, SSH with proper key mounting)
- `enabled`: Boolean to enable/disable this repository
- `schedule`: Cron expression for pull schedule

### Docker Compose Configuration

The `docker-compose.yml` file manages:
- Port mapping (default 5000)
- Volume management for configuration and data
- Restart policies
- Networking

To modify port or other settings, edit `docker-compose.yml`:

```yaml
ports:
  - "8080:5000"  # Change external port to 8080
```

## Advanced Configuration

### SSH Repositories

For private repositories using SSH:

1. Mount your SSH key:

```yaml
volumes:
  - ~/.ssh:/root/.ssh:ro
  - ./config:/config:ro
  - markmywords_data:/data
```

2. Use SSH URL in configuration:

```json
{
  "private-repo": {
    "url": "git@github.com:username/private-repo.git",
    "enabled": true,
    "schedule": "0 * * * *"
  }
}
```

### Custom Port

Edit `docker-compose.yml`:

```yaml
ports:
  - "3000:5000"
```

### Persistent Data

The Docker volume `markmywords_data` stores:
- Cloned repositories (`/data/repos`)
- Search index (`/data/search_index`)

To backup data:

```bash
docker-compose exec markmywords tar czf - /data > markmywords_backup.tar.gz
```

## API Reference

### GET /
Main page with repository list and search interface

### GET /view/<repo>/<path>
View a markdown file as HTML

**Parameters**:
- `repo`: Repository name
- `path`: Path to markdown file within the repository

**Example**: `/view/my-wiki/docs/getting-started.md`

### GET /api/search
Search across all markdown files

**Query Parameters**:
- `q`: Search query (minimum 2 characters)

**Response**:
```json
{
  "results": [
    {
      "path": "repo-name/path/to/file.md",
      "title": "File Title"
    }
  ]
}
```

### GET /api/status
Get application status and repository information

**Response**:
```json
{
  "status": "running",
  "repositories": {
    "repo-name": {
      "cloned": true,
      "last_modified": "2024-01-15T10:30:00"
    }
  },
  "timestamp": "2024-01-15T11:00:00"
}
```

## Troubleshooting

### Repositories Not Updating

1. Check configuration file syntax:
```bash
docker-compose exec markmywords cat /config/repositories.json
```

2. View application logs:
```bash
docker-compose logs -f markmywords
```

3. Verify git access:
```bash
docker-compose exec markmywords git clone <url> /tmp/test
```

### Search Not Working

1. Ensure markdown files are present:
```bash
docker-compose exec markmywords ls -la /data/repos/
```

2. Check search index:
```bash
docker-compose exec markmywords ls -la /data/search_index/
```

3. Rebuild search index by restarting:
```bash
docker-compose restart markmywords
```

### Port Already in Use

Change the port in `docker-compose.yml`:

```yaml
ports:
  - "5001:5000"
```

Then run:
```bash
docker-compose up -d
```

## Permissions

- Git operations require read access to repositories
- For private repositories, ensure proper SSH key setup
- Web service runs as root inside container (safe for self-hosted)

## Security Considerations

- Only use HTTPS git URLs or secure SSH keys
- Mount SSH keys as read-only
- Use firewall rules to restrict access
- Consider running behind a reverse proxy (nginx)
- No authentication built-in; add via reverse proxy if needed

## Performance

- Search uses Whoosh full-text index
- Index built on startup and updated after each pull
- Markdown rendering uses Python markdown library with extensions

## Architecture

```
┌─────────────────────────────────┐
│        markMyWords              │
│  (Flask + APScheduler)          │
├─────────────────────────────────┤
│ Scheduled Git Pulls (cron)      │
│ Markdown → HTML Conversion      │
│ Full-text Search (Whoosh)       │
├─────────────────────────────────┤
│        Docker Volume            │
│ - Cloned Repositories           │
│ - Search Index                  │
└─────────────────────────────────┘
```

## Development

### Local Setup

```bash
# Create virtual environment
python -m venv venv
source venv/bin/activate

# Install dependencies
pip install -r requirements.txt

# Create config
mkdir -p config
cp config/repositories.json.example config/repositories.json

# Run locally
python app.py
```

### Requirements

- Python 3.11+
- Flask 3.0+
- APScheduler 3.10+
- markdown 3.5+
- Whoosh 2.7+

## License

MIT License - See LICENSE file for details

## Contributing

Contributions welcome! Please submit issues and pull requests.

## Support

For issues, questions, or suggestions, please open an issue on GitHub.