SHA256
Initial vibe-coded commit.
This commit is contained in:
Discsearcher
+46
@@ -0,0 +1,46 @@
|
||||
# Python
|
||||
__pycache__/
|
||||
*.py[cod]
|
||||
*$py.class
|
||||
*.so
|
||||
.Python
|
||||
build/
|
||||
develop-eggs/
|
||||
dist/
|
||||
downloads/
|
||||
eggs/
|
||||
.eggs/
|
||||
lib/
|
||||
lib64/
|
||||
parts/
|
||||
sdist/
|
||||
var/
|
||||
wheels/
|
||||
*.egg-info/
|
||||
.installed.cfg
|
||||
*.egg
|
||||
MANIFEST
|
||||
venv/
|
||||
ENV/
|
||||
env/
|
||||
.venv
|
||||
|
||||
# IDE
|
||||
.vscode/
|
||||
.idea/
|
||||
*.swp
|
||||
*.swo
|
||||
*~
|
||||
.DS_Store
|
||||
|
||||
# Project-specific
|
||||
/data/
|
||||
/config/repositories.json
|
||||
*.db
|
||||
*.sqlite
|
||||
.env
|
||||
.env.local
|
||||
|
||||
# Docker
|
||||
.dockerignore
|
||||
docker-compose.override.yml
|
||||
+28
@@ -0,0 +1,28 @@
|
||||
FROM python:3.11-slim
|
||||
|
||||
WORKDIR /app
|
||||
|
||||
# Install git (required for cloning and pulling repositories)
|
||||
RUN apt-get update && apt-get install -y git && rm -rf /var/lib/apt/lists/*
|
||||
|
||||
# Copy requirements and install dependencies
|
||||
COPY requirements.txt .
|
||||
RUN pip install --no-cache-dir -r requirements.txt
|
||||
|
||||
# Copy application code
|
||||
COPY app.py .
|
||||
COPY templates/ ./templates/
|
||||
COPY static/ ./static/
|
||||
|
||||
# Create necessary directories
|
||||
RUN mkdir -p /data /config
|
||||
|
||||
# Expose port
|
||||
EXPOSE 5000
|
||||
|
||||
# Health check
|
||||
HEALTHCHECK --interval=30s --timeout=10s --start-period=5s --retries=3 \
|
||||
CMD python -c "import urllib.request; urllib.request.urlopen('http://localhost:5000/api/status').read()"
|
||||
|
||||
# Run the application
|
||||
CMD ["python", "app.py"]
|
||||
@@ -0,0 +1,21 @@
|
||||
MIT License
|
||||
|
||||
Copyright (c) 2024 markMyWords Contributors
|
||||
|
||||
Permission is hereby granted, free of charge, to any person obtaining a copy
|
||||
of this software and associated documentation files (the "Software"), to deal
|
||||
in the Software without restriction, including without limitation the rights
|
||||
to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
|
||||
copies of the Software, and to permit persons to whom the Software is
|
||||
furnished to do so, subject to the following conditions:
|
||||
|
||||
The above copyright notice and this permission notice shall be included in all
|
||||
copies or substantial portions of the Software.
|
||||
|
||||
THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
|
||||
IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
|
||||
FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
|
||||
AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
|
||||
LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
|
||||
OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE
|
||||
SOFTWARE.
|
||||
@@ -0,0 +1,328 @@
|
||||
# markMyWords
|
||||
|
||||
A self-hostable Docker application that automatically pulls markdown repositories at user-defined intervals and displays them as searchable HTML pages.
|
||||
|
||||
## Features
|
||||
|
||||
- 🚀 **Self-hosted**: Run entirely on your own infrastructure
|
||||
- 📦 **Docker-ready**: Simple Docker Compose setup
|
||||
- 📅 **Scheduled Pulls**: Configure cron-style schedules for each repository
|
||||
- 🔍 **Full-text Search**: Search across all markdown files
|
||||
- 📄 **Markdown Rendering**: Beautiful HTML rendering with syntax highlighting
|
||||
- ⚡ **Zero Configuration**: Works out of the box with configuration file
|
||||
|
||||
## Quick Start
|
||||
|
||||
### 1. Clone and Configure
|
||||
|
||||
```bash
|
||||
cd markmywords
|
||||
# Create the config directory
|
||||
mkdir -p config
|
||||
|
||||
# Copy the example configuration
|
||||
cp config/repositories.json.example config/repositories.json
|
||||
|
||||
# Edit the configuration with your repositories
|
||||
nano config/repositories.json
|
||||
```
|
||||
|
||||
### 2. Configure Repositories
|
||||
|
||||
Edit `config/repositories.json` to specify which repositories to pull:
|
||||
|
||||
```json
|
||||
{
|
||||
"repo-name": {
|
||||
"url": "https://github.com/username/repo.git",
|
||||
"enabled": true,
|
||||
"schedule": "0 */6 * * *"
|
||||
}
|
||||
}
|
||||
```
|
||||
|
||||
**Schedule Format**: Standard cron expression (minute, hour, day of month, month, day of week)
|
||||
|
||||
Common schedules:
|
||||
- `0 * * * *` - Every hour
|
||||
- `0 */6 * * *` - Every 6 hours (default)
|
||||
- `0 9 * * *` - Daily at 9 AM
|
||||
- `0 0 * * 0` - Weekly (Sunday at midnight)
|
||||
|
||||
### 3. Run with Docker Compose
|
||||
|
||||
```bash
|
||||
docker-compose up -d
|
||||
```
|
||||
|
||||
The application will be available at `http://localhost:5000`
|
||||
|
||||
### 4. Access the Application
|
||||
|
||||
- **Main Page**: http://localhost:5000
|
||||
- View all repositories and their markdown files
|
||||
- Search across all files
|
||||
|
||||
- **View File**: Click on any markdown file to view it as HTML
|
||||
|
||||
- **API Endpoints**:
|
||||
- `/api/status` - Application status and repository info
|
||||
- `/api/search?q=<query>` - Search markdown files
|
||||
|
||||
## Configuration
|
||||
|
||||
### repositories.json
|
||||
|
||||
```json
|
||||
{
|
||||
"repo-name": {
|
||||
"url": "https://github.com/username/repo.git",
|
||||
"enabled": true,
|
||||
"schedule": "0 */6 * * *"
|
||||
}
|
||||
}
|
||||
```
|
||||
|
||||
**Fields**:
|
||||
- `url`: Git repository URL (HTTPS recommended, SSH with proper key mounting)
|
||||
- `enabled`: Boolean to enable/disable this repository
|
||||
- `schedule`: Cron expression for pull schedule
|
||||
|
||||
### Docker Compose Configuration
|
||||
|
||||
The `docker-compose.yml` file manages:
|
||||
- Port mapping (default 5000)
|
||||
- Volume management for configuration and data
|
||||
- Restart policies
|
||||
- Networking
|
||||
|
||||
To modify port or other settings, edit `docker-compose.yml`:
|
||||
|
||||
```yaml
|
||||
ports:
|
||||
- "8080:5000" # Change external port to 8080
|
||||
```
|
||||
|
||||
## Advanced Configuration
|
||||
|
||||
### SSH Repositories
|
||||
|
||||
For private repositories using SSH:
|
||||
|
||||
1. Mount your SSH key:
|
||||
|
||||
```yaml
|
||||
volumes:
|
||||
- ~/.ssh:/root/.ssh:ro
|
||||
- ./config:/config:ro
|
||||
- markmywords_data:/data
|
||||
```
|
||||
|
||||
2. Use SSH URL in configuration:
|
||||
|
||||
```json
|
||||
{
|
||||
"private-repo": {
|
||||
"url": "git@github.com:username/private-repo.git",
|
||||
"enabled": true,
|
||||
"schedule": "0 * * * *"
|
||||
}
|
||||
}
|
||||
```
|
||||
|
||||
### Custom Port
|
||||
|
||||
Edit `docker-compose.yml`:
|
||||
|
||||
```yaml
|
||||
ports:
|
||||
- "3000:5000"
|
||||
```
|
||||
|
||||
### Persistent Data
|
||||
|
||||
The Docker volume `markmywords_data` stores:
|
||||
- Cloned repositories (`/data/repos`)
|
||||
- Search index (`/data/search_index`)
|
||||
|
||||
To backup data:
|
||||
|
||||
```bash
|
||||
docker-compose exec markmywords tar czf - /data > markmywords_backup.tar.gz
|
||||
```
|
||||
|
||||
## API Reference
|
||||
|
||||
### GET /
|
||||
Main page with repository list and search interface
|
||||
|
||||
### GET /view/<repo>/<path>
|
||||
View a markdown file as HTML
|
||||
|
||||
**Parameters**:
|
||||
- `repo`: Repository name
|
||||
- `path`: Path to markdown file within the repository
|
||||
|
||||
**Example**: `/view/my-wiki/docs/getting-started.md`
|
||||
|
||||
### GET /api/search
|
||||
Search across all markdown files
|
||||
|
||||
**Query Parameters**:
|
||||
- `q`: Search query (minimum 2 characters)
|
||||
|
||||
**Response**:
|
||||
```json
|
||||
{
|
||||
"results": [
|
||||
{
|
||||
"path": "repo-name/path/to/file.md",
|
||||
"title": "File Title"
|
||||
}
|
||||
]
|
||||
}
|
||||
```
|
||||
|
||||
### GET /api/status
|
||||
Get application status and repository information
|
||||
|
||||
**Response**:
|
||||
```json
|
||||
{
|
||||
"status": "running",
|
||||
"repositories": {
|
||||
"repo-name": {
|
||||
"cloned": true,
|
||||
"last_modified": "2024-01-15T10:30:00"
|
||||
}
|
||||
},
|
||||
"timestamp": "2024-01-15T11:00:00"
|
||||
}
|
||||
```
|
||||
|
||||
## Troubleshooting
|
||||
|
||||
### Repositories Not Updating
|
||||
|
||||
1. Check configuration file syntax:
|
||||
```bash
|
||||
docker-compose exec markmywords cat /config/repositories.json
|
||||
```
|
||||
|
||||
2. View application logs:
|
||||
```bash
|
||||
docker-compose logs -f markmywords
|
||||
```
|
||||
|
||||
3. Verify git access:
|
||||
```bash
|
||||
docker-compose exec markmywords git clone <url> /tmp/test
|
||||
```
|
||||
|
||||
### Search Not Working
|
||||
|
||||
1. Ensure markdown files are present:
|
||||
```bash
|
||||
docker-compose exec markmywords ls -la /data/repos/
|
||||
```
|
||||
|
||||
2. Check search index:
|
||||
```bash
|
||||
docker-compose exec markmywords ls -la /data/search_index/
|
||||
```
|
||||
|
||||
3. Rebuild search index by restarting:
|
||||
```bash
|
||||
docker-compose restart markmywords
|
||||
```
|
||||
|
||||
### Port Already in Use
|
||||
|
||||
Change the port in `docker-compose.yml`:
|
||||
|
||||
```yaml
|
||||
ports:
|
||||
- "5001:5000"
|
||||
```
|
||||
|
||||
Then run:
|
||||
```bash
|
||||
docker-compose up -d
|
||||
```
|
||||
|
||||
## Permissions
|
||||
|
||||
- Git operations require read access to repositories
|
||||
- For private repositories, ensure proper SSH key setup
|
||||
- Web service runs as root inside container (safe for self-hosted)
|
||||
|
||||
## Security Considerations
|
||||
|
||||
- Only use HTTPS git URLs or secure SSH keys
|
||||
- Mount SSH keys as read-only
|
||||
- Use firewall rules to restrict access
|
||||
- Consider running behind a reverse proxy (nginx)
|
||||
- No authentication built-in; add via reverse proxy if needed
|
||||
|
||||
## Performance
|
||||
|
||||
- Search uses Whoosh full-text index
|
||||
- Index built on startup and updated after each pull
|
||||
- Markdown rendering uses Python markdown library with extensions
|
||||
|
||||
## Architecture
|
||||
|
||||
```
|
||||
┌─────────────────────────────────┐
|
||||
│ markMyWords │
|
||||
│ (Flask + APScheduler) │
|
||||
├─────────────────────────────────┤
|
||||
│ Scheduled Git Pulls (cron) │
|
||||
│ Markdown → HTML Conversion │
|
||||
│ Full-text Search (Whoosh) │
|
||||
├─────────────────────────────────┤
|
||||
│ Docker Volume │
|
||||
│ - Cloned Repositories │
|
||||
│ - Search Index │
|
||||
└─────────────────────────────────┘
|
||||
```
|
||||
|
||||
## Development
|
||||
|
||||
### Local Setup
|
||||
|
||||
```bash
|
||||
# Create virtual environment
|
||||
python -m venv venv
|
||||
source venv/bin/activate
|
||||
|
||||
# Install dependencies
|
||||
pip install -r requirements.txt
|
||||
|
||||
# Create config
|
||||
mkdir -p config
|
||||
cp config/repositories.json.example config/repositories.json
|
||||
|
||||
# Run locally
|
||||
python app.py
|
||||
```
|
||||
|
||||
### Requirements
|
||||
|
||||
- Python 3.11+
|
||||
- Flask 3.0+
|
||||
- APScheduler 3.10+
|
||||
- markdown 3.5+
|
||||
- Whoosh 2.7+
|
||||
|
||||
## License
|
||||
|
||||
MIT License - See LICENSE file for details
|
||||
|
||||
## Contributing
|
||||
|
||||
Contributions welcome! Please submit issues and pull requests.
|
||||
|
||||
## Support
|
||||
|
||||
For issues, questions, or suggestions, please open an issue on GitHub.
|
||||
@@ -0,0 +1,299 @@
|
||||
#!/usr/bin/env python3
|
||||
"""
|
||||
markmywords - Self-hosted markdown repository viewer with search
|
||||
"""
|
||||
|
||||
import os
|
||||
import subprocess
|
||||
import json
|
||||
from pathlib import Path
|
||||
from datetime import datetime
|
||||
from flask import Flask, render_template, request, jsonify
|
||||
from apscheduler.schedulers.background import BackgroundScheduler
|
||||
from apscheduler.triggers.cron import CronTrigger
|
||||
import markdown
|
||||
import logging
|
||||
from whoosh.index import create_in, open_dir
|
||||
from whoosh.fields import Schema, TEXT, ID
|
||||
from whoosh.qparser import QueryParser
|
||||
|
||||
app = Flask(__name__)
|
||||
|
||||
# Configuration
|
||||
CONFIG_FILE = "/config/repositories.json"
|
||||
REPOS_DIR = "/data/repos"
|
||||
INDEX_DIR = "/data/search_index"
|
||||
MARKDOWN_EXTENSIONS = ['.md', '.markdown']
|
||||
|
||||
# Setup logging
|
||||
logging.basicConfig(level=logging.INFO)
|
||||
logger = logging.getLogger(__name__)
|
||||
|
||||
# Ensure directories exist
|
||||
os.makedirs(REPOS_DIR, exist_ok=True)
|
||||
os.makedirs(INDEX_DIR, exist_ok=True)
|
||||
|
||||
# Initialize search index
|
||||
def create_search_index():
|
||||
"""Create or open the search index."""
|
||||
schema = Schema(
|
||||
path=ID(stored=True),
|
||||
title=TEXT(stored=True),
|
||||
content=TEXT(stored=False)
|
||||
)
|
||||
|
||||
if not os.path.exists(os.path.join(INDEX_DIR, "WRITELOCK")):
|
||||
return create_in(INDEX_DIR, schema)
|
||||
return open_dir(INDEX_DIR)
|
||||
|
||||
ix = create_search_index()
|
||||
|
||||
def load_repositories():
|
||||
"""Load repository configuration from file."""
|
||||
if not os.path.exists(CONFIG_FILE):
|
||||
logger.warning(f"Config file not found: {CONFIG_FILE}")
|
||||
return {}
|
||||
|
||||
try:
|
||||
with open(CONFIG_FILE, 'r') as f:
|
||||
return json.load(f)
|
||||
except Exception as e:
|
||||
logger.error(f"Error loading config: {e}")
|
||||
return {}
|
||||
|
||||
def git_pull_repo(repo_name, repo_path):
|
||||
"""Perform a git pull on the specified repository."""
|
||||
try:
|
||||
logger.info(f"Pulling repository: {repo_name}")
|
||||
result = subprocess.run(
|
||||
["git", "pull"],
|
||||
cwd=repo_path,
|
||||
capture_output=True,
|
||||
text=True,
|
||||
timeout=300
|
||||
)
|
||||
logger.info(f"Pull result for {repo_name}: {result.stdout}")
|
||||
if result.returncode != 0:
|
||||
logger.error(f"Pull error for {repo_name}: {result.stderr}")
|
||||
return result.returncode == 0
|
||||
except subprocess.TimeoutExpired:
|
||||
logger.error(f"Git pull timeout for {repo_name}")
|
||||
return False
|
||||
except Exception as e:
|
||||
logger.error(f"Error pulling {repo_name}: {e}")
|
||||
return False
|
||||
|
||||
def clone_or_pull(repo_name, repo_url):
|
||||
"""Clone repository if it doesn't exist, otherwise pull."""
|
||||
repo_path = os.path.join(REPOS_DIR, repo_name)
|
||||
|
||||
if not os.path.exists(repo_path):
|
||||
try:
|
||||
logger.info(f"Cloning repository: {repo_name} from {repo_url}")
|
||||
subprocess.run(
|
||||
["git", "clone", repo_url, repo_path],
|
||||
capture_output=True,
|
||||
text=True,
|
||||
timeout=300
|
||||
)
|
||||
logger.info(f"Successfully cloned {repo_name}")
|
||||
except Exception as e:
|
||||
logger.error(f"Error cloning {repo_name}: {e}")
|
||||
return False
|
||||
else:
|
||||
return git_pull_repo(repo_name, repo_path)
|
||||
|
||||
return True
|
||||
|
||||
def update_search_index():
|
||||
"""Update the search index with all markdown files."""
|
||||
try:
|
||||
writer = ix.writer()
|
||||
|
||||
for repo_dir in Path(REPOS_DIR).iterdir():
|
||||
if not repo_dir.is_dir():
|
||||
continue
|
||||
|
||||
for md_file in repo_dir.rglob('*'):
|
||||
if md_file.suffix.lower() in MARKDOWN_EXTENSIONS:
|
||||
try:
|
||||
content = md_file.read_text(encoding='utf-8', errors='ignore')
|
||||
# Extract title from filename or first heading
|
||||
title = md_file.stem
|
||||
rel_path = str(md_file.relative_to(REPOS_DIR))
|
||||
|
||||
writer.add_document(
|
||||
path=rel_path,
|
||||
title=title,
|
||||
content=content
|
||||
)
|
||||
except Exception as e:
|
||||
logger.error(f"Error indexing {md_file}: {e}")
|
||||
|
||||
writer.commit()
|
||||
logger.info("Search index updated successfully")
|
||||
except Exception as e:
|
||||
logger.error(f"Error updating search index: {e}")
|
||||
|
||||
def scheduled_pull():
|
||||
"""Perform scheduled pulls of all repositories."""
|
||||
logger.info("Starting scheduled repository pull")
|
||||
repos = load_repositories()
|
||||
|
||||
for repo_name, repo_config in repos.items():
|
||||
if not repo_config.get('enabled', True):
|
||||
continue
|
||||
|
||||
clone_or_pull(repo_name, repo_config['url'])
|
||||
|
||||
# Update search index after pulling
|
||||
update_search_index()
|
||||
logger.info("Scheduled pull completed")
|
||||
|
||||
def setup_scheduler():
|
||||
"""Setup the background scheduler for periodic pulls."""
|
||||
scheduler = BackgroundScheduler()
|
||||
repos = load_repositories()
|
||||
|
||||
for repo_name, repo_config in repos.items():
|
||||
if repo_config.get('enabled', True):
|
||||
cron_schedule = repo_config.get('schedule', '0 */6 * * *') # Default: every 6 hours
|
||||
try:
|
||||
scheduler.add_job(
|
||||
scheduled_pull,
|
||||
CronTrigger.from_crontab(cron_schedule),
|
||||
id=f"pull_{repo_name}",
|
||||
name=f"Pull {repo_name}"
|
||||
)
|
||||
logger.info(f"Scheduled pull for {repo_name}: {cron_schedule}")
|
||||
except Exception as e:
|
||||
logger.error(f"Error scheduling {repo_name}: {e}")
|
||||
|
||||
scheduler.start()
|
||||
return scheduler
|
||||
|
||||
# Routes
|
||||
@app.route('/')
|
||||
def index():
|
||||
"""Display the main page with list of repositories and search."""
|
||||
repos = load_repositories()
|
||||
repo_list = []
|
||||
|
||||
for repo_name in repos.keys():
|
||||
repo_path = os.path.join(REPOS_DIR, repo_name)
|
||||
markdown_files = []
|
||||
|
||||
if os.path.exists(repo_path):
|
||||
for md_file in Path(repo_path).rglob('*'):
|
||||
if md_file.suffix.lower() in MARKDOWN_EXTENSIONS:
|
||||
rel_path = str(md_file.relative_to(repo_path))
|
||||
markdown_files.append({
|
||||
'name': md_file.stem,
|
||||
'path': rel_path
|
||||
})
|
||||
|
||||
repo_list.append({
|
||||
'name': repo_name,
|
||||
'files': sorted(markdown_files, key=lambda x: x['path'])
|
||||
})
|
||||
|
||||
return render_template('index.html', repositories=repo_list)
|
||||
|
||||
@app.route('/api/search', methods=['GET'])
|
||||
def search():
|
||||
"""Search markdown files."""
|
||||
query_str = request.args.get('q', '').strip()
|
||||
|
||||
if not query_str or len(query_str) < 2:
|
||||
return jsonify({'results': []})
|
||||
|
||||
try:
|
||||
with ix.searcher() as searcher:
|
||||
query = QueryParser("content", ix.schema).parse(query_str)
|
||||
results = searcher.search(query)
|
||||
|
||||
search_results = []
|
||||
for hit in results:
|
||||
search_results.append({
|
||||
'path': hit['path'],
|
||||
'title': hit['title']
|
||||
})
|
||||
|
||||
return jsonify({'results': search_results})
|
||||
except Exception as e:
|
||||
logger.error(f"Search error: {e}")
|
||||
return jsonify({'results': [], 'error': str(e)})
|
||||
|
||||
@app.route('/view/<repo>/<path:filepath>')
|
||||
def view_file(repo, filepath):
|
||||
"""View a markdown file converted to HTML."""
|
||||
# Security: prevent directory traversal
|
||||
if '..' in filepath or filepath.startswith('/'):
|
||||
return "Invalid path", 400
|
||||
|
||||
file_path = os.path.join(REPOS_DIR, repo, filepath)
|
||||
|
||||
# Ensure the file is within the repo directory
|
||||
try:
|
||||
file_path = os.path.realpath(file_path)
|
||||
repo_path = os.path.realpath(os.path.join(REPOS_DIR, repo))
|
||||
if not file_path.startswith(repo_path):
|
||||
return "Access denied", 403
|
||||
except Exception:
|
||||
return "Invalid path", 400
|
||||
|
||||
if not os.path.exists(file_path):
|
||||
return "File not found", 404
|
||||
|
||||
try:
|
||||
with open(file_path, 'r', encoding='utf-8') as f:
|
||||
md_content = f.read()
|
||||
|
||||
# Convert markdown to HTML
|
||||
html_content = markdown.markdown(
|
||||
md_content,
|
||||
extensions=['extra', 'codehilite', 'toc']
|
||||
)
|
||||
|
||||
return render_template(
|
||||
'view.html',
|
||||
repo=repo,
|
||||
filepath=filepath,
|
||||
content=html_content,
|
||||
filename=os.path.basename(filepath)
|
||||
)
|
||||
except Exception as e:
|
||||
logger.error(f"Error reading file {file_path}: {e}")
|
||||
return f"Error reading file: {e}", 500
|
||||
|
||||
@app.route('/api/status')
|
||||
def status():
|
||||
"""Return application status."""
|
||||
repos = load_repositories()
|
||||
repo_status = {}
|
||||
|
||||
for repo_name in repos.keys():
|
||||
repo_path = os.path.join(REPOS_DIR, repo_name)
|
||||
repo_status[repo_name] = {
|
||||
'cloned': os.path.exists(repo_path),
|
||||
'last_modified': datetime.fromtimestamp(
|
||||
os.path.getmtime(repo_path)
|
||||
).isoformat() if os.path.exists(repo_path) else None
|
||||
}
|
||||
|
||||
return jsonify({
|
||||
'status': 'running',
|
||||
'repositories': repo_status,
|
||||
'timestamp': datetime.now().isoformat()
|
||||
})
|
||||
|
||||
if __name__ == '__main__':
|
||||
# Perform initial pull and index
|
||||
logger.info("Initializing markmywords")
|
||||
scheduled_pull()
|
||||
|
||||
# Setup scheduler
|
||||
scheduler = setup_scheduler()
|
||||
|
||||
# Start Flask app
|
||||
app.run(host='0.0.0.0', port=5000, debug=False)
|
||||
@@ -0,0 +1,12 @@
|
||||
{
|
||||
"my-wiki": {
|
||||
"url": "https://github.com/username/my-wiki.git",
|
||||
"enabled": true,
|
||||
"schedule": "0 */6 * * *"
|
||||
},
|
||||
"project-docs": {
|
||||
"url": "https://github.com/username/project-docs.git",
|
||||
"enabled": true,
|
||||
"schedule": "0 9 * * *"
|
||||
}
|
||||
}
|
||||
@@ -0,0 +1,29 @@
|
||||
# Optional override file for local development
|
||||
# Copy to docker-compose.override.yml and customize
|
||||
|
||||
version: '3.8'
|
||||
|
||||
services:
|
||||
markmywords:
|
||||
# Enable debug mode
|
||||
environment:
|
||||
FLASK_ENV: development
|
||||
FLASK_DEBUG: 1
|
||||
|
||||
# Mount SSH keys for private repositories
|
||||
volumes:
|
||||
- ~/.ssh:/root/.ssh:ro
|
||||
|
||||
# Port forwarding examples
|
||||
# ports:
|
||||
# - "8080:5000" # Different external port
|
||||
|
||||
# Additional resource limits
|
||||
# deploy:
|
||||
# resources:
|
||||
# limits:
|
||||
# cpus: '0.5'
|
||||
# memory: 512M
|
||||
# reservations:
|
||||
# cpus: '0.25'
|
||||
# memory: 256M
|
||||
@@ -0,0 +1,29 @@
|
||||
version: '3.8'
|
||||
|
||||
services:
|
||||
markmywords:
|
||||
build: .
|
||||
container_name: markmywords
|
||||
ports:
|
||||
- "5000:5000"
|
||||
volumes:
|
||||
# Configuration directory with repositories.json
|
||||
- ./config:/config:ro
|
||||
# Data directory for cloned repositories and search index
|
||||
- markmywords_data:/data
|
||||
environment:
|
||||
# Set Flask environment
|
||||
FLASK_ENV: production
|
||||
# Python unbuffered output for real-time logs
|
||||
PYTHONUNBUFFERED: 1
|
||||
restart: unless-stopped
|
||||
networks:
|
||||
- markmywords
|
||||
|
||||
networks:
|
||||
markmywords:
|
||||
driver: bridge
|
||||
|
||||
volumes:
|
||||
markmywords_data:
|
||||
driver: local
|
||||
@@ -0,0 +1,4 @@
|
||||
Flask==3.0.0
|
||||
APScheduler==3.10.4
|
||||
markdown==3.5.2
|
||||
Whoosh==2.7.4
|
||||
@@ -0,0 +1,77 @@
|
||||
// Search functionality for markmywords
|
||||
|
||||
function performSearch() {
|
||||
const searchInput = document.getElementById('searchInput');
|
||||
const query = searchInput.value.trim();
|
||||
|
||||
if (query.length < 2) {
|
||||
alert('Please enter at least 2 characters to search');
|
||||
return;
|
||||
}
|
||||
|
||||
fetch(`/api/search?q=${encodeURIComponent(query)}`)
|
||||
.then(response => response.json())
|
||||
.then(data => {
|
||||
displaySearchResults(data.results, query);
|
||||
})
|
||||
.catch(error => {
|
||||
console.error('Search error:', error);
|
||||
alert('An error occurred during search');
|
||||
});
|
||||
}
|
||||
|
||||
function displaySearchResults(results, query) {
|
||||
const searchResultsDiv = document.getElementById('searchResults');
|
||||
const resultsList = document.getElementById('resultsList');
|
||||
|
||||
resultsList.innerHTML = '';
|
||||
|
||||
if (results.length === 0) {
|
||||
resultsList.innerHTML = '<p>No results found for "' + escapeHtml(query) + '"</p>';
|
||||
searchResultsDiv.style.display = 'block';
|
||||
return;
|
||||
}
|
||||
|
||||
results.forEach(result => {
|
||||
const resultItem = document.createElement('div');
|
||||
resultItem.className = 'result-item';
|
||||
|
||||
const link = document.createElement('a');
|
||||
link.href = `/view/${result.path}`;
|
||||
link.textContent = result.title || result.path;
|
||||
|
||||
const pathSpan = document.createElement('div');
|
||||
pathSpan.className = 'result-path';
|
||||
pathSpan.textContent = result.path;
|
||||
|
||||
resultItem.appendChild(link);
|
||||
resultItem.appendChild(pathSpan);
|
||||
|
||||
resultsList.appendChild(resultItem);
|
||||
});
|
||||
|
||||
searchResultsDiv.style.display = 'block';
|
||||
}
|
||||
|
||||
function escapeHtml(text) {
|
||||
const map = {
|
||||
'&': '&',
|
||||
'<': '<',
|
||||
'>': '>',
|
||||
'"': '"',
|
||||
"'": '''
|
||||
};
|
||||
return text.replace(/[&<>"']/g, m => map[m]);
|
||||
}
|
||||
|
||||
// Allow search on Enter key
|
||||
document.addEventListener('DOMContentLoaded', function() {
|
||||
const searchInput = document.getElementById('searchInput');
|
||||
if (searchInput) {
|
||||
searchInput.addEventListener('keypress', function(event) {
|
||||
if (event.key === 'Enter') {
|
||||
performSearch();
|
||||
}
|
||||
});
|
||||
}
|
||||
});
|
||||
@@ -0,0 +1,427 @@
|
||||
:root {
|
||||
--primary-color: #2c3e50;
|
||||
--secondary-color: #3498db;
|
||||
--accent-color: #e74c3c;
|
||||
--background-color: #ecf0f1;
|
||||
--text-color: #2c3e50;
|
||||
--border-color: #bdc3c7;
|
||||
--success-color: #27ae60;
|
||||
}
|
||||
|
||||
* {
|
||||
margin: 0;
|
||||
padding: 0;
|
||||
box-sizing: border-box;
|
||||
}
|
||||
|
||||
html, body {
|
||||
height: 100%;
|
||||
}
|
||||
|
||||
body {
|
||||
font-family: -apple-system, BlinkMacSystemFont, 'Segoe UI', Roboto, Oxygen, Ubuntu, Cantarell, sans-serif;
|
||||
color: var(--text-color);
|
||||
background-color: var(--background-color);
|
||||
display: flex;
|
||||
flex-direction: column;
|
||||
}
|
||||
|
||||
/* Navbar */
|
||||
.navbar {
|
||||
background-color: var(--primary-color);
|
||||
color: white;
|
||||
padding: 1rem 0;
|
||||
box-shadow: 0 2px 4px rgba(0,0,0,0.1);
|
||||
}
|
||||
|
||||
.navbar-container {
|
||||
max-width: 1200px;
|
||||
margin: 0 auto;
|
||||
padding: 0 1rem;
|
||||
display: flex;
|
||||
justify-content: space-between;
|
||||
align-items: center;
|
||||
}
|
||||
|
||||
.navbar-logo {
|
||||
font-size: 1.5rem;
|
||||
font-weight: bold;
|
||||
color: white;
|
||||
text-decoration: none;
|
||||
transition: color 0.3s ease;
|
||||
}
|
||||
|
||||
.navbar-logo:hover {
|
||||
color: var(--secondary-color);
|
||||
}
|
||||
|
||||
.navbar-menu {
|
||||
display: flex;
|
||||
gap: 2rem;
|
||||
}
|
||||
|
||||
.nav-link {
|
||||
color: white;
|
||||
text-decoration: none;
|
||||
transition: color 0.3s ease;
|
||||
}
|
||||
|
||||
.nav-link:hover {
|
||||
color: var(--secondary-color);
|
||||
}
|
||||
|
||||
/* Main Content */
|
||||
main {
|
||||
flex: 1;
|
||||
padding: 2rem 1rem;
|
||||
}
|
||||
|
||||
.container {
|
||||
max-width: 1200px;
|
||||
margin: 0 auto;
|
||||
}
|
||||
|
||||
/* Search Section */
|
||||
.search-section {
|
||||
background-color: white;
|
||||
padding: 2rem;
|
||||
border-radius: 8px;
|
||||
box-shadow: 0 2px 4px rgba(0,0,0,0.1);
|
||||
margin-bottom: 2rem;
|
||||
}
|
||||
|
||||
.search-section h1 {
|
||||
margin-bottom: 1.5rem;
|
||||
color: var(--primary-color);
|
||||
}
|
||||
|
||||
.search-box {
|
||||
display: flex;
|
||||
gap: 0.5rem;
|
||||
margin-bottom: 1.5rem;
|
||||
}
|
||||
|
||||
.search-input {
|
||||
flex: 1;
|
||||
padding: 0.75rem 1rem;
|
||||
border: 2px solid var(--border-color);
|
||||
border-radius: 4px;
|
||||
font-size: 1rem;
|
||||
transition: border-color 0.3s ease;
|
||||
}
|
||||
|
||||
.search-input:focus {
|
||||
outline: none;
|
||||
border-color: var(--secondary-color);
|
||||
}
|
||||
|
||||
.search-button {
|
||||
padding: 0.75rem 1.5rem;
|
||||
background-color: var(--secondary-color);
|
||||
color: white;
|
||||
border: none;
|
||||
border-radius: 4px;
|
||||
cursor: pointer;
|
||||
font-size: 1rem;
|
||||
font-weight: 500;
|
||||
transition: background-color 0.3s ease;
|
||||
}
|
||||
|
||||
.search-button:hover {
|
||||
background-color: #2980b9;
|
||||
}
|
||||
|
||||
/* Search Results */
|
||||
.search-results {
|
||||
margin-top: 2rem;
|
||||
padding-top: 1.5rem;
|
||||
border-top: 2px solid var(--border-color);
|
||||
}
|
||||
|
||||
.search-results h2 {
|
||||
margin-bottom: 1rem;
|
||||
color: var(--primary-color);
|
||||
}
|
||||
|
||||
#resultsList {
|
||||
display: grid;
|
||||
gap: 1rem;
|
||||
}
|
||||
|
||||
.result-item {
|
||||
padding: 1rem;
|
||||
background-color: #f8f9fa;
|
||||
border-left: 4px solid var(--secondary-color);
|
||||
border-radius: 4px;
|
||||
transition: transform 0.2s ease, box-shadow 0.2s ease;
|
||||
}
|
||||
|
||||
.result-item:hover {
|
||||
transform: translateX(4px);
|
||||
box-shadow: 0 2px 8px rgba(0,0,0,0.1);
|
||||
}
|
||||
|
||||
.result-item a {
|
||||
color: var(--secondary-color);
|
||||
text-decoration: none;
|
||||
font-weight: 500;
|
||||
}
|
||||
|
||||
.result-item a:hover {
|
||||
text-decoration: underline;
|
||||
}
|
||||
|
||||
.result-path {
|
||||
font-size: 0.9rem;
|
||||
color: #7f8c8d;
|
||||
margin-top: 0.5rem;
|
||||
}
|
||||
|
||||
/* Repositories Section */
|
||||
.repositories-section {
|
||||
margin-top: 2rem;
|
||||
}
|
||||
|
||||
.repositories-section h2 {
|
||||
margin-bottom: 1.5rem;
|
||||
color: var(--primary-color);
|
||||
}
|
||||
|
||||
.repo-grid {
|
||||
display: grid;
|
||||
grid-template-columns: repeat(auto-fill, minmax(300px, 1fr));
|
||||
gap: 1.5rem;
|
||||
}
|
||||
|
||||
.repo-card {
|
||||
background-color: white;
|
||||
padding: 1.5rem;
|
||||
border-radius: 8px;
|
||||
box-shadow: 0 2px 4px rgba(0,0,0,0.1);
|
||||
transition: transform 0.3s ease, box-shadow 0.3s ease;
|
||||
}
|
||||
|
||||
.repo-card:hover {
|
||||
transform: translateY(-4px);
|
||||
box-shadow: 0 4px 12px rgba(0,0,0,0.15);
|
||||
}
|
||||
|
||||
.repo-card h3 {
|
||||
color: var(--primary-color);
|
||||
margin-bottom: 1rem;
|
||||
border-bottom: 2px solid var(--secondary-color);
|
||||
padding-bottom: 0.5rem;
|
||||
}
|
||||
|
||||
.file-list ul {
|
||||
list-style: none;
|
||||
}
|
||||
|
||||
.file-list li {
|
||||
padding: 0.5rem 0;
|
||||
}
|
||||
|
||||
.file-link {
|
||||
color: var(--secondary-color);
|
||||
text-decoration: none;
|
||||
transition: color 0.2s ease;
|
||||
}
|
||||
|
||||
.file-link:hover {
|
||||
color: #2980b9;
|
||||
text-decoration: underline;
|
||||
}
|
||||
|
||||
.no-files {
|
||||
color: #7f8c8d;
|
||||
font-style: italic;
|
||||
}
|
||||
|
||||
.empty-state {
|
||||
text-align: center;
|
||||
padding: 3rem;
|
||||
background-color: white;
|
||||
border-radius: 8px;
|
||||
color: #7f8c8d;
|
||||
}
|
||||
|
||||
/* Breadcrumb */
|
||||
.breadcrumb {
|
||||
margin-bottom: 1.5rem;
|
||||
padding: 1rem;
|
||||
background-color: #f8f9fa;
|
||||
border-radius: 4px;
|
||||
font-size: 0.95rem;
|
||||
}
|
||||
|
||||
.breadcrumb a {
|
||||
color: var(--secondary-color);
|
||||
text-decoration: none;
|
||||
transition: color 0.2s ease;
|
||||
}
|
||||
|
||||
.breadcrumb a:hover {
|
||||
color: #2980b9;
|
||||
text-decoration: underline;
|
||||
}
|
||||
|
||||
.breadcrumb-repo,
|
||||
.breadcrumb-file {
|
||||
color: var(--text-color);
|
||||
}
|
||||
|
||||
/* Markdown Content */
|
||||
.markdown-content {
|
||||
background-color: white;
|
||||
padding: 2rem;
|
||||
border-radius: 8px;
|
||||
box-shadow: 0 2px 4px rgba(0,0,0,0.1);
|
||||
line-height: 1.8;
|
||||
}
|
||||
|
||||
.markdown-content h1 {
|
||||
color: var(--primary-color);
|
||||
margin-bottom: 1.5rem;
|
||||
border-bottom: 3px solid var(--secondary-color);
|
||||
padding-bottom: 0.5rem;
|
||||
}
|
||||
|
||||
.markdown-content h2 {
|
||||
color: var(--primary-color);
|
||||
margin-top: 1.5rem;
|
||||
margin-bottom: 1rem;
|
||||
}
|
||||
|
||||
.markdown-content h3,
|
||||
.markdown-content h4,
|
||||
.markdown-content h5,
|
||||
.markdown-content h6 {
|
||||
color: var(--primary-color);
|
||||
margin-top: 1rem;
|
||||
margin-bottom: 0.5rem;
|
||||
}
|
||||
|
||||
.markdown-content p {
|
||||
margin-bottom: 1rem;
|
||||
}
|
||||
|
||||
.markdown-content ul,
|
||||
.markdown-content ol {
|
||||
margin-left: 1.5rem;
|
||||
margin-bottom: 1rem;
|
||||
}
|
||||
|
||||
.markdown-content li {
|
||||
margin-bottom: 0.5rem;
|
||||
}
|
||||
|
||||
.markdown-content a {
|
||||
color: var(--secondary-color);
|
||||
text-decoration: none;
|
||||
transition: color 0.2s ease;
|
||||
}
|
||||
|
||||
.markdown-content a:hover {
|
||||
color: #2980b9;
|
||||
text-decoration: underline;
|
||||
}
|
||||
|
||||
.markdown-content code {
|
||||
background-color: #f8f9fa;
|
||||
padding: 0.2rem 0.4rem;
|
||||
border-radius: 3px;
|
||||
font-family: 'Courier New', monospace;
|
||||
font-size: 0.9em;
|
||||
color: #c7254e;
|
||||
}
|
||||
|
||||
.markdown-content pre {
|
||||
background-color: #2c3e50;
|
||||
color: #ecf0f1;
|
||||
padding: 1rem;
|
||||
border-radius: 4px;
|
||||
overflow-x: auto;
|
||||
margin-bottom: 1rem;
|
||||
}
|
||||
|
||||
.markdown-content pre code {
|
||||
background-color: transparent;
|
||||
color: inherit;
|
||||
padding: 0;
|
||||
}
|
||||
|
||||
.markdown-content blockquote {
|
||||
border-left: 4px solid var(--secondary-color);
|
||||
padding-left: 1rem;
|
||||
margin: 1rem 0;
|
||||
color: #7f8c8d;
|
||||
font-style: italic;
|
||||
}
|
||||
|
||||
.markdown-content table {
|
||||
width: 100%;
|
||||
border-collapse: collapse;
|
||||
margin-bottom: 1rem;
|
||||
}
|
||||
|
||||
.markdown-content th,
|
||||
.markdown-content td {
|
||||
border: 1px solid var(--border-color);
|
||||
padding: 0.75rem;
|
||||
text-align: left;
|
||||
}
|
||||
|
||||
.markdown-content th {
|
||||
background-color: #f8f9fa;
|
||||
font-weight: bold;
|
||||
}
|
||||
|
||||
.markdown-content img {
|
||||
max-width: 100%;
|
||||
height: auto;
|
||||
border-radius: 4px;
|
||||
margin: 1rem 0;
|
||||
}
|
||||
|
||||
/* Footer */
|
||||
.footer {
|
||||
background-color: var(--primary-color);
|
||||
color: white;
|
||||
text-align: center;
|
||||
padding: 1.5rem;
|
||||
margin-top: 2rem;
|
||||
}
|
||||
|
||||
.footer p {
|
||||
margin: 0;
|
||||
font-size: 0.9rem;
|
||||
}
|
||||
|
||||
/* Responsive Design */
|
||||
@media (max-width: 768px) {
|
||||
.navbar-container {
|
||||
flex-direction: column;
|
||||
gap: 1rem;
|
||||
}
|
||||
|
||||
.navbar-menu {
|
||||
gap: 1rem;
|
||||
}
|
||||
|
||||
.search-box {
|
||||
flex-direction: column;
|
||||
}
|
||||
|
||||
.repo-grid {
|
||||
grid-template-columns: 1fr;
|
||||
}
|
||||
|
||||
.search-section,
|
||||
.markdown-content {
|
||||
padding: 1rem;
|
||||
}
|
||||
|
||||
.markdown-content {
|
||||
padding: 1rem;
|
||||
}
|
||||
}
|
||||
@@ -0,0 +1,28 @@
|
||||
<!DOCTYPE html>
|
||||
<html lang="en">
|
||||
<head>
|
||||
<meta charset="UTF-8">
|
||||
<meta name="viewport" content="width=device-width, initial-scale=1.0">
|
||||
<title>{% block title %}markMyWords{% endblock %}</title>
|
||||
<link rel="stylesheet" href="{{ url_for('static', filename='style.css') }}">
|
||||
{% block extra_head %}{% endblock %}
|
||||
</head>
|
||||
<body>
|
||||
<nav class="navbar">
|
||||
<div class="navbar-container">
|
||||
<a href="/" class="navbar-logo">markMyWords</a>
|
||||
<div class="navbar-menu">
|
||||
<a href="/" class="nav-link">Home</a>
|
||||
</div>
|
||||
</div>
|
||||
</nav>
|
||||
|
||||
<main class="main-content">
|
||||
{% block content %}{% endblock %}
|
||||
</main>
|
||||
|
||||
<footer class="footer">
|
||||
<p>© 2024 markMyWords - Self-hosted Markdown Repository Viewer</p>
|
||||
</footer>
|
||||
</body>
|
||||
</html>
|
||||
@@ -0,0 +1,62 @@
|
||||
{% extends "base.html" %}
|
||||
|
||||
{% block title %}Home - markMyWords{% endblock %}
|
||||
|
||||
{% block extra_head %}
|
||||
<script src="{{ url_for('static', filename='search.js') }}"></script>
|
||||
{% endblock %}
|
||||
|
||||
{% block content %}
|
||||
<div class="container">
|
||||
<section class="search-section">
|
||||
<h1>Search Markdown Files</h1>
|
||||
<div class="search-box">
|
||||
<input
|
||||
type="text"
|
||||
id="searchInput"
|
||||
class="search-input"
|
||||
placeholder="Search across all repositories..."
|
||||
autocomplete="off"
|
||||
>
|
||||
<button class="search-button" onclick="performSearch()">Search</button>
|
||||
</div>
|
||||
|
||||
<div id="searchResults" class="search-results" style="display: none;">
|
||||
<h2>Search Results</h2>
|
||||
<div id="resultsList"></div>
|
||||
</div>
|
||||
</section>
|
||||
|
||||
<section class="repositories-section">
|
||||
<h2>Repositories</h2>
|
||||
{% if repositories %}
|
||||
<div class="repo-grid">
|
||||
{% for repo in repositories %}
|
||||
<div class="repo-card">
|
||||
<h3>{{ repo.name }}</h3>
|
||||
{% if repo.files %}
|
||||
<div class="file-list">
|
||||
<ul>
|
||||
{% for file in repo.files %}
|
||||
<li>
|
||||
<a href="/view/{{ repo.name }}/{{ file.path }}" class="file-link">
|
||||
📄 {{ file.name }}
|
||||
</a>
|
||||
</li>
|
||||
{% endfor %}
|
||||
</ul>
|
||||
</div>
|
||||
{% else %}
|
||||
<p class="no-files">No markdown files found</p>
|
||||
{% endif %}
|
||||
</div>
|
||||
{% endfor %}
|
||||
</div>
|
||||
{% else %}
|
||||
<div class="empty-state">
|
||||
<p>No repositories configured. Please check your configuration file.</p>
|
||||
</div>
|
||||
{% endif %}
|
||||
</section>
|
||||
</div>
|
||||
{% endblock %}
|
||||
@@ -0,0 +1,20 @@
|
||||
{% extends "base.html" %}
|
||||
|
||||
{% block title %}{{ filename }} - markMyWords{% endblock %}
|
||||
|
||||
{% block content %}
|
||||
<div class="container">
|
||||
<div class="breadcrumb">
|
||||
<a href="/">Home</a> >
|
||||
<span class="breadcrumb-repo">{{ repo }}</span> >
|
||||
<span class="breadcrumb-file">{{ filename }}</span>
|
||||
</div>
|
||||
|
||||
<article class="markdown-content">
|
||||
<h1>{{ filename }}</h1>
|
||||
<div class="content-body">
|
||||
{{ content|safe }}
|
||||
</div>
|
||||
</article>
|
||||
</div>
|
||||
{% endblock %}
|
||||
Reference in New Issue
Block a user