Simon Willison has released version 1.11 of shot-scraper, a Python tool designed for web scraping and data extraction. The update focuses on command option consistency and improves the reliability of the server mechanism used across the video and multi subcommands. Previously, these functions would fail if the target server took longer than a single second to start accepting traffic. The new logic now polls for port availability and waits up to thirty seconds before timing out.
Developers can now load JavaScript from a local file, standard input, or a GitHub repository using the –js-file option. This change replaces the older –javascript argument which required passing code directly as a string. The multi subcommand also supports a js_file key within YAML configuration files. Additionally, the javascript and html commands have received a –timeout option to align with other available flags.
* Polling replaces the fixed one-second delay for server connections.
* Local JavaScript files are supported via –js-file or js_file keys.
* Timeout options are consistent across javascript and html commands.
