Config Reference ================ This page documents how configuration values are resolved and what each configuration key does. Precedence ---------- The effective precedence is: 1. CLI Parameter 2. Environment Variable 3. Config File 4. Default (Host) 5. Default Default (host) values come from ``web_novel_scraper/resources/host_config.json`` and have some default settings for different hosts like using FlareSolverr or a bigger timeout. Config File Location Precedence ------------------------------- The location of the config file itself has its own precedence: 1. ``config_file`` CLI Parameter 2. ``SCRAPER_CONFIG_FILE`` Environment Variable 3. Internal default path (``$HOME/.config/web-novel-scraper/config.json`` on Linux) If file loading fails (missing, empty, malformed), the scraper falls back to an empty file config and continues with the rest of the precedence chain. Configuration Schema -------------------- Top-level keys: - ``base_novels_dir`` (string) - ``decode_guide_file`` (string) - ``request_config`` (object) ``request_config`` keys: - ``force_flaresolver`` (bool) - ``flaresolver_url`` (string) - ``request_timeout`` (int) - ``request_retries`` (int) - ``request_time_between_retries`` (int) - ``request_cookies`` (dict) Options Detail -------------- ``base_novels_dir`` ~~~~~~~~~~~~~~~~~~~ - Type: string path - Default: ``platformdirs.user_data_dir("web-novel-scraper", "web-novel-scraper")`` - Purpose: base directory where novel folders are stored. - Sources: - ``parameter: base_novels_dir`` - ``env: SCRAPER_BASE_NOVELS_DIR`` - ``config file: base_novels_dir`` ``decode_guide_file`` ~~~~~~~~~~~~~~~~~~~~~ - Type: string path - Default: bundled decode guide path ``web_novel_scraper/decode_guide/decode_guide.json`` - Purpose: file used to load host extraction rules. - Sources: - ``parameter: decode_guide_file`` - ``env: SCRAPER_DECODE_GUIDE_FILE`` - ``config file: decode_guide_file`` ``request_config.force_flaresolver`` ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ - Type: bool - Default: ``false`` - Purpose: force requests through FlareSolverr. - Accepted bool-like values: ``true/false``, ``1/0``, ``yes/no`` (case-insensitive), plus native bool/int. - Sources: - ``parameter: force_flaresolver`` - ``env: SCRAPER_FORCE_FLARESOLVER`` - ``config file: request_config.force_flaresolver`` - ``default(host): request_config.force_flaresolver`` ``request_config.flaresolver_url`` ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ - Type: string URL - Default: ``http://localhost:8191`` - Purpose: FlareSolverr endpoint. - Sources: - ``env: SCRAPER_FLARESOLVER_URL`` - ``config file: request_config.flaresolver_url`` ``request_config.request_timeout`` ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ - Type: int - Default: ``20`` - Purpose: request timeout in seconds. - Sources: - ``parameter: request_timeout`` - ``env: SCRAPER_REQUEST_TIMEOUT`` - ``config file: request_config.request_timeout`` - ``default(host): request_config.request_timeout`` ``request_config.request_retries`` ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ - Type: int - Default: ``3`` - Purpose: max retries per request. - Sources: - ``parameter: request_retries`` - ``env: SCRAPER_REQUEST_RETRIES`` - ``config file: request_config.request_retries`` - ``default(host): request_config.request_retries`` ``request_config.request_time_between_retries`` ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ - Type: int - Default: ``3`` - Purpose: seconds to wait between retries if request fails. - Sources: - ``parameter: request_time_between_retries`` - ``env: SCRAPER_REQUEST_TIME_BETWEEN_RETRIES`` - ``config file: request_config.request_time_between_retries`` - ``default(host): request_config.request_time_between_retries`` ``request_config.request_cookies`` ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ - Type: dict - Default: ``{}`` - Purpose: cookies sent with requests, useful for sites requiring specific session or preference cookies. - Sources: - ``config file: request_config.request_cookies`` - ``default(host): request_config.request_cookies`` Minimal Config File Example --------------------------- .. code-block:: json { "base_novels_dir": "/home/user/novels", "decode_guide_file": "/home/user/custom_decode_guide.json", "request_config": { "force_flaresolver": true, "flaresolver_url": "http://localhost:8191", "request_timeout": 30, "request_retries": 5, "request_time_between_retries": 4, "request_cookies": { "custom_toc": "10000" } } }