feat: add OpenInsider Top-of-day crawler and multi-CRAWLER_TYPE support

New crawler: app/crawlers/openinsider_top.py\n- Scrapes three pages (sales/purchases/officer purchases)\n- Filters rows with Value/Amount >= ,000,000 (configurable via INSIDER_MIN_AMOUNT)\n- Builds concise notifications; saves to data/openinsider_top.json

Runner: support comma-separated CRAWLER_TYPE and new openinsider_top type\n- Accepts e.g., CRAWLER_TYPE=openinsider_top,openinsider,barrons\n- Preserves order, removes duplicates; warns on unknown types\n- Uses shared schedule: RUN_DAILY_AT or CHECK_INTERVAL; initial run per crawler

Entrypoint: rename enhanced_crawler.py -> main.py\n- Update Dockerfile CMD and README references

Config & docs:\n- Reorganize .env.template into clear sections with examples\n- Update .env with multi-crawler example and INSIDER_MIN_AMOUNT\n- README: document new crawler, usage, and multi-type CRAWLER_TYPE
This commit is contained in:
2025-09-09 21:17:50 +08:00
parent b2c58c0560
commit f708f3bf1d
6 changed files with 308 additions and 27 deletions

View File

@@ -1,10 +1,19 @@
# 基本設定
#############################
# 基本設定(排程與日誌)
#############################
# 每 N 秒檢查一次(若設定 RUN_DAILY_AT 則忽略)
CHECK_INTERVAL=300
# 每天固定時間檢查(例如 12:00若不需固定時間請保留此值或註解
RUN_DAILY_AT=12:00
# 日誌等級DEBUG | INFO | WARNING | ERROR
LOG_LEVEL=INFO
# 啟動後第一次是否也發送目前清單true/false
ALWAYS_NOTIFY_ON_STARTUP=false
# 電子郵件通知設定Gmail 範例)
#############################
# 通知設定(擇一或多個)
#############################
# Email範例以 Gmail
EMAIL_SMTP_SERVER=smtp.gmail.com
EMAIL_SMTP_PORT=587
# 可選: starttls | ssl | none
@@ -20,15 +29,27 @@ WEBHOOK_URL=https://hooks.slack.com/services/YOUR/SLACK/WEBHOOK
# Discord Webhook可選
DISCORD_WEBHOOK=https://discord.com/api/webhooks/YOUR/DISCORD/WEBHOOK
# 進階(可選):自訂資料與日誌目錄
# 預設 Docker 會使用 /app/data、/app/logs本機則使用 ./data、./logs
#############################
# 進階:資料與日誌路徑(可選)
#############################
# Docker 預設 /app/data、/app/logs本機預設 ./data、./logs
# DATA_DIR=./data
# LOG_DIR=./logs
# 選擇爬蟲類型與參數
# 可選: barrons | openinsider
#############################
# 爬蟲選擇與參數
#############################
# 可選barrons | openinsider | openinsider_top
# 可逗號同時啟多個(例如同時跑三種):
# CRAWLER_TYPE=openinsider_top,openinsider,barrons
CRAWLER_TYPE=openinsider
# 針對內部人交易爬蟲的股票代號(單一)
# OpenInsider依個別股票查詢
# - 單一標的使用 SYMBOL
# - 多個標的使用 SYMBOLS以逗號分隔
SYMBOL=PLTR
# 或一次追多個:以逗號分隔
# SYMBOLS=PLTR,NVDA,TSLA
# OpenInsider 當日大額(三頁合併)
# - 金額門檻(整數,單位:美元),僅通知 >= 該金額的交易
# INSIDER_MIN_AMOUNT=1000000