SHA256
Initial vibe-coded commit.
This commit is contained in:
Discsearcher
@@ -0,0 +1,328 @@
|
||||
# markMyWords
|
||||
|
||||
A self-hostable Docker application that automatically pulls markdown repositories at user-defined intervals and displays them as searchable HTML pages.
|
||||
|
||||
## Features
|
||||
|
||||
- 🚀 **Self-hosted**: Run entirely on your own infrastructure
|
||||
- 📦 **Docker-ready**: Simple Docker Compose setup
|
||||
- 📅 **Scheduled Pulls**: Configure cron-style schedules for each repository
|
||||
- 🔍 **Full-text Search**: Search across all markdown files
|
||||
- 📄 **Markdown Rendering**: Beautiful HTML rendering with syntax highlighting
|
||||
- ⚡ **Zero Configuration**: Works out of the box with configuration file
|
||||
|
||||
## Quick Start
|
||||
|
||||
### 1. Clone and Configure
|
||||
|
||||
```bash
|
||||
cd markmywords
|
||||
# Create the config directory
|
||||
mkdir -p config
|
||||
|
||||
# Copy the example configuration
|
||||
cp config/repositories.json.example config/repositories.json
|
||||
|
||||
# Edit the configuration with your repositories
|
||||
nano config/repositories.json
|
||||
```
|
||||
|
||||
### 2. Configure Repositories
|
||||
|
||||
Edit `config/repositories.json` to specify which repositories to pull:
|
||||
|
||||
```json
|
||||
{
|
||||
"repo-name": {
|
||||
"url": "https://github.com/username/repo.git",
|
||||
"enabled": true,
|
||||
"schedule": "0 */6 * * *"
|
||||
}
|
||||
}
|
||||
```
|
||||
|
||||
**Schedule Format**: Standard cron expression (minute, hour, day of month, month, day of week)
|
||||
|
||||
Common schedules:
|
||||
- `0 * * * *` - Every hour
|
||||
- `0 */6 * * *` - Every 6 hours (default)
|
||||
- `0 9 * * *` - Daily at 9 AM
|
||||
- `0 0 * * 0` - Weekly (Sunday at midnight)
|
||||
|
||||
### 3. Run with Docker Compose
|
||||
|
||||
```bash
|
||||
docker-compose up -d
|
||||
```
|
||||
|
||||
The application will be available at `http://localhost:5000`
|
||||
|
||||
### 4. Access the Application
|
||||
|
||||
- **Main Page**: http://localhost:5000
|
||||
- View all repositories and their markdown files
|
||||
- Search across all files
|
||||
|
||||
- **View File**: Click on any markdown file to view it as HTML
|
||||
|
||||
- **API Endpoints**:
|
||||
- `/api/status` - Application status and repository info
|
||||
- `/api/search?q=<query>` - Search markdown files
|
||||
|
||||
## Configuration
|
||||
|
||||
### repositories.json
|
||||
|
||||
```json
|
||||
{
|
||||
"repo-name": {
|
||||
"url": "https://github.com/username/repo.git",
|
||||
"enabled": true,
|
||||
"schedule": "0 */6 * * *"
|
||||
}
|
||||
}
|
||||
```
|
||||
|
||||
**Fields**:
|
||||
- `url`: Git repository URL (HTTPS recommended, SSH with proper key mounting)
|
||||
- `enabled`: Boolean to enable/disable this repository
|
||||
- `schedule`: Cron expression for pull schedule
|
||||
|
||||
### Docker Compose Configuration
|
||||
|
||||
The `docker-compose.yml` file manages:
|
||||
- Port mapping (default 5000)
|
||||
- Volume management for configuration and data
|
||||
- Restart policies
|
||||
- Networking
|
||||
|
||||
To modify port or other settings, edit `docker-compose.yml`:
|
||||
|
||||
```yaml
|
||||
ports:
|
||||
- "8080:5000" # Change external port to 8080
|
||||
```
|
||||
|
||||
## Advanced Configuration
|
||||
|
||||
### SSH Repositories
|
||||
|
||||
For private repositories using SSH:
|
||||
|
||||
1. Mount your SSH key:
|
||||
|
||||
```yaml
|
||||
volumes:
|
||||
- ~/.ssh:/root/.ssh:ro
|
||||
- ./config:/config:ro
|
||||
- markmywords_data:/data
|
||||
```
|
||||
|
||||
2. Use SSH URL in configuration:
|
||||
|
||||
```json
|
||||
{
|
||||
"private-repo": {
|
||||
"url": "git@github.com:username/private-repo.git",
|
||||
"enabled": true,
|
||||
"schedule": "0 * * * *"
|
||||
}
|
||||
}
|
||||
```
|
||||
|
||||
### Custom Port
|
||||
|
||||
Edit `docker-compose.yml`:
|
||||
|
||||
```yaml
|
||||
ports:
|
||||
- "3000:5000"
|
||||
```
|
||||
|
||||
### Persistent Data
|
||||
|
||||
The Docker volume `markmywords_data` stores:
|
||||
- Cloned repositories (`/data/repos`)
|
||||
- Search index (`/data/search_index`)
|
||||
|
||||
To backup data:
|
||||
|
||||
```bash
|
||||
docker-compose exec markmywords tar czf - /data > markmywords_backup.tar.gz
|
||||
```
|
||||
|
||||
## API Reference
|
||||
|
||||
### GET /
|
||||
Main page with repository list and search interface
|
||||
|
||||
### GET /view/<repo>/<path>
|
||||
View a markdown file as HTML
|
||||
|
||||
**Parameters**:
|
||||
- `repo`: Repository name
|
||||
- `path`: Path to markdown file within the repository
|
||||
|
||||
**Example**: `/view/my-wiki/docs/getting-started.md`
|
||||
|
||||
### GET /api/search
|
||||
Search across all markdown files
|
||||
|
||||
**Query Parameters**:
|
||||
- `q`: Search query (minimum 2 characters)
|
||||
|
||||
**Response**:
|
||||
```json
|
||||
{
|
||||
"results": [
|
||||
{
|
||||
"path": "repo-name/path/to/file.md",
|
||||
"title": "File Title"
|
||||
}
|
||||
]
|
||||
}
|
||||
```
|
||||
|
||||
### GET /api/status
|
||||
Get application status and repository information
|
||||
|
||||
**Response**:
|
||||
```json
|
||||
{
|
||||
"status": "running",
|
||||
"repositories": {
|
||||
"repo-name": {
|
||||
"cloned": true,
|
||||
"last_modified": "2024-01-15T10:30:00"
|
||||
}
|
||||
},
|
||||
"timestamp": "2024-01-15T11:00:00"
|
||||
}
|
||||
```
|
||||
|
||||
## Troubleshooting
|
||||
|
||||
### Repositories Not Updating
|
||||
|
||||
1. Check configuration file syntax:
|
||||
```bash
|
||||
docker-compose exec markmywords cat /config/repositories.json
|
||||
```
|
||||
|
||||
2. View application logs:
|
||||
```bash
|
||||
docker-compose logs -f markmywords
|
||||
```
|
||||
|
||||
3. Verify git access:
|
||||
```bash
|
||||
docker-compose exec markmywords git clone <url> /tmp/test
|
||||
```
|
||||
|
||||
### Search Not Working
|
||||
|
||||
1. Ensure markdown files are present:
|
||||
```bash
|
||||
docker-compose exec markmywords ls -la /data/repos/
|
||||
```
|
||||
|
||||
2. Check search index:
|
||||
```bash
|
||||
docker-compose exec markmywords ls -la /data/search_index/
|
||||
```
|
||||
|
||||
3. Rebuild search index by restarting:
|
||||
```bash
|
||||
docker-compose restart markmywords
|
||||
```
|
||||
|
||||
### Port Already in Use
|
||||
|
||||
Change the port in `docker-compose.yml`:
|
||||
|
||||
```yaml
|
||||
ports:
|
||||
- "5001:5000"
|
||||
```
|
||||
|
||||
Then run:
|
||||
```bash
|
||||
docker-compose up -d
|
||||
```
|
||||
|
||||
## Permissions
|
||||
|
||||
- Git operations require read access to repositories
|
||||
- For private repositories, ensure proper SSH key setup
|
||||
- Web service runs as root inside container (safe for self-hosted)
|
||||
|
||||
## Security Considerations
|
||||
|
||||
- Only use HTTPS git URLs or secure SSH keys
|
||||
- Mount SSH keys as read-only
|
||||
- Use firewall rules to restrict access
|
||||
- Consider running behind a reverse proxy (nginx)
|
||||
- No authentication built-in; add via reverse proxy if needed
|
||||
|
||||
## Performance
|
||||
|
||||
- Search uses Whoosh full-text index
|
||||
- Index built on startup and updated after each pull
|
||||
- Markdown rendering uses Python markdown library with extensions
|
||||
|
||||
## Architecture
|
||||
|
||||
```
|
||||
┌─────────────────────────────────┐
|
||||
│ markMyWords │
|
||||
│ (Flask + APScheduler) │
|
||||
├─────────────────────────────────┤
|
||||
│ Scheduled Git Pulls (cron) │
|
||||
│ Markdown → HTML Conversion │
|
||||
│ Full-text Search (Whoosh) │
|
||||
├─────────────────────────────────┤
|
||||
│ Docker Volume │
|
||||
│ - Cloned Repositories │
|
||||
│ - Search Index │
|
||||
└─────────────────────────────────┘
|
||||
```
|
||||
|
||||
## Development
|
||||
|
||||
### Local Setup
|
||||
|
||||
```bash
|
||||
# Create virtual environment
|
||||
python -m venv venv
|
||||
source venv/bin/activate
|
||||
|
||||
# Install dependencies
|
||||
pip install -r requirements.txt
|
||||
|
||||
# Create config
|
||||
mkdir -p config
|
||||
cp config/repositories.json.example config/repositories.json
|
||||
|
||||
# Run locally
|
||||
python app.py
|
||||
```
|
||||
|
||||
### Requirements
|
||||
|
||||
- Python 3.11+
|
||||
- Flask 3.0+
|
||||
- APScheduler 3.10+
|
||||
- markdown 3.5+
|
||||
- Whoosh 2.7+
|
||||
|
||||
## License
|
||||
|
||||
MIT License - See LICENSE file for details
|
||||
|
||||
## Contributing
|
||||
|
||||
Contributions welcome! Please submit issues and pull requests.
|
||||
|
||||
## Support
|
||||
|
||||
For issues, questions, or suggestions, please open an issue on GitHub.
|
||||
Reference in New Issue
Block a user