From 418964fa91b1c44e8f6e5fc323932f952cd558b7 Mon Sep 17 00:00:00 2001 From: pukkandan Date: Tue, 10 Aug 2021 20:10:39 +0530 Subject: [PATCH] Release 2021.08.10 --- .github/FUNDING.yml | 2 +- CONTRIBUTORS | 11 +++++++++ Changelog.md | 54 +++++++++++++++++++++++++++++++++++++++++++++ Collaborators.md | 2 ++ Makefile | 2 +- README.md | 14 ++++++------ supportedsites.md | 9 ++++++++ 7 files changed, 85 insertions(+), 9 deletions(-) diff --git a/.github/FUNDING.yml b/.github/FUNDING.yml index ddfcd32c1e..04de087085 100644 --- a/.github/FUNDING.yml +++ b/.github/FUNDING.yml @@ -10,4 +10,4 @@ liberapay: # Replace with a single Liberapay username issuehunt: # Replace with a single IssueHunt username otechie: # Replace with a single Otechie username -custom: ['https://github.com/yt-dlp/yt-dlp/blob/master/Collaborators.md'] +custom: ['https://github.com/yt-dlp/yt-dlp/blob/master/Collaborators.md#collaborators'] diff --git a/CONTRIBUTORS b/CONTRIBUTORS index 58019ee503..fe28dfc784 100644 --- a/CONTRIBUTORS +++ b/CONTRIBUTORS @@ -67,3 +67,14 @@ zerodytrash wesnm pento rigstot +dirkf +funniray +Jessecar96 +jhwgh1968 +kikuyan +max-te +nchilada +pgaig +PSlava +stdedos +u-spec-png diff --git a/Changelog.md b/Changelog.md index d07480b5af..21876d9444 100644 --- a/Changelog.md +++ b/Changelog.md @@ -19,6 +19,60 @@ # Instuctions for creating release --> +### 2021.08.10 + +* Add option `--replace-in-metadata` +* Add option `--no-simulate` to not simulate even when `--print` or `--list...` are used - Deprecates `--print-json` +* Allow entire infodict to be printed using `%()s` - makes `--dump-json` redundant +* Allow multiple `--exec` and `--exec-before-download` +* Add regex to `--match-filter` +* Add all format filtering operators also to `--match-filter` by [max-te](https://github.com/max-te) +* Add compat-option `no-keep-subs` +* [adobepass] Add MSO Cablevision by [Jessecar96](https://github.com/Jessecar96) +* [BandCamp] Add BandcampMusicIE by [Ashish0804](https://github.com/Ashish0804) +* [blackboardcollaborate] Add new extractor by [Ashish0804](https://github.com/Ashish0804) +* [eroprofile] Add album downloader by [jhwgh1968](https://github.com/jhwgh1968) +* [mirrativ] Add extractors by [nao20010128nao](https://github.com/nao20010128nao) +* [openrec] Add extractors by [nao20010128nao](https://github.com/nao20010128nao) +* [nbcolympics:stream] Fix extractor by [nchilada](https://github.com/nchilada), [pukkandan](https://github.com/pukkandan) +* [nbcolympics] Update extractor for 2020 olympics by [wesnm](https://github.com/wesnm) +* [paramountplus] Separate extractor and fix some titles by [shirt](https://github.com/shirt-dev), [pukkandan](https://github.com/pukkandan) +* [RCTIPlus] Support events and TV by [MinePlayersPE](https://github.com/MinePlayersPE) +* [Newgrounds] Improve extractor and fix playlist by [u-spec-png](https://github.com/u-spec-png) +* [aenetworks] Update `_THEPLATFORM_KEY` and `_THEPLATFORM_SECRET` by [wesnm](https://github.com/wesnm) +* [crunchyroll] Fix thumbnail by [funniray](https://github.com/funniray) +* [HotStar] Use API for metadata and extract subtitles by [Ashish0804](https://github.com/Ashish0804) +* [instagram] Fix comments extraction by [u-spec-png](https://github.com/u-spec-png) +* [peertube] Fix videos without description by [u-spec-png](https://github.com/u-spec-png) +* [twitch:clips] Extract `display_id` by [dirkf](https://github.com/dirkf) +* [viki] Print error message from API request +* [Vine] Remove invalid formats by [u-spec-png](https://github.com/u-spec-png) +* [VrtNU] Fix XSRF token by [pgaig](https://github.com/pgaig) +* [vrv] Fix thumbnail extraction by [funniray](https://github.com/funniray) +* [youtube] Add extractor-arg `include-live-dash` to show live dash formats +* [youtube] Improve signature function detection by [PSlava](https://github.com/PSlava) +* [youtube] Raise appropriate error when API pages can't be downloaded +* Ensure `_write_ytdl_file` closes file handle on error +* Fix `--compat-options filename` by [stdedos](https://github.com/stdedos) +* Fix issues with infodict sanitization +* Fix resuming when using `--no-part` +* Fix wrong extension for intermediate files +* Handle `BrokenPipeError` by [kikuyan](https://github.com/kikuyan) +* Show libraries present in verbose head +* [extractor] Detect `sttp` as subtitles in MPD by [fstirlitz](https://github.com/fstirlitz) +* [extractor] Reset non-repeating warnings per video +* [ffmpeg] Fix streaming `mp4` to `stdout` +* [ffpmeg] Allow `--ffmpeg-location` to be a file with different name +* [utils] Fix `InAdvancePagedList.__getitem__` +* [utils] Fix `traverse_obj` depth when `is_user_input` +* [webvtt] Merge daisy-chained duplicate cues by [fstirlitz](https://github.com/fstirlitz) +* [build] Use custom build of `pyinstaller` by [shirt](https://github.com/shirt-dev) +* [tests:download] Add batch testing for extractors (`test_YourExtractor_all`) +* [docs] Document which fields `--add-metadata` adds to the file +* [docs] Fix some mistakes and improve doc +* [cleanup] Misc code cleanup + + ### 2021.08.02 * Add logo, banner and donate links diff --git a/Collaborators.md b/Collaborators.md index a657044275..0017e1cd48 100644 --- a/Collaborators.md +++ b/Collaborators.md @@ -15,6 +15,8 @@ ## [pukkandan](https://github.com/pukkandan) ## [shirt](https://github.com/shirt-dev) +[![ko-fi](https://img.shields.io/badge/_-Ko--fi-red.svg?logo=kofi&labelColor=555555&style=for-the-badge)](https://ko-fi.com/shirt) + * Multithreading (`-N`) and aria2c support for fragment downloads * Support for media initialization and discontinuity in HLS * The self-updater (`-U`) diff --git a/Makefile b/Makefile index fb7b8a0cce..bc0b4e399e 100644 --- a/Makefile +++ b/Makefile @@ -13,7 +13,7 @@ pypi-files: AUTHORS Changelog.md LICENSE README.md README.txt supportedsites com .PHONY: all clean install test tar pypi-files completions ot offlinetest codetest supportedsites clean-test: - rm -rf *.dump *.part* *.ytdl *.info.json *.mp4 *.m4a *.flv *.mp3 *.avi *.mkv *.webm *.3gp *.wav *.ape *.swf *.jpg *.png *.frag *.frag.urls *.frag.aria2 test/testdata/player-*.js + rm -rf *.dump *.part* *.ytdl *.info.json *.mp4 *.m4a *.flv *.mp3 *.avi *.mkv *.webm *.3gp *.wav *.ape *.swf *.jpg *.png *.frag *.frag.urls *.frag.aria2 test/testdata/player-*.js *.opus *.webp *.ttml *.vtt *.jpeg clean-dist: rm -rf yt-dlp.1.temp.md yt-dlp.1 README.txt MANIFEST build/ dist/ .coverage cover/ yt-dlp.tar.gz completions/ yt_dlp/extractor/lazy_extractors.py *.spec CONTRIBUTING.md.tmp yt-dlp yt-dlp.exe yt_dlp.egg-info/ AUTHORS .mailmap clean-cache: diff --git a/README.md b/README.md index 493f437d1b..dcd870decb 100644 --- a/README.md +++ b/README.md @@ -5,7 +5,7 @@ [![Release version](https://img.shields.io/github/v/release/yt-dlp/yt-dlp?color=blue&label=&style=for-the-badge)](https://github.com/yt-dlp/yt-dlp/releases/latest) [![CI Status](https://img.shields.io/github/workflow/status/yt-dlp/yt-dlp/Core%20Tests/master?label=&style=for-the-badge)](https://github.com/yt-dlp/yt-dlp/actions) [![License: Unlicense](https://img.shields.io/badge/-Unlicense-blue.svg?style=for-the-badge)](LICENSE) -[![Donate](https://img.shields.io/badge/_-Donate-red.svg?logo=githubsponsors&labelColor=555555&style=for-the-badge)](Collaborators.md) +[![Donate](https://img.shields.io/badge/_-Donate-red.svg?logo=githubsponsors&labelColor=555555&style=for-the-badge)](Collaborators.md#collaborators) [![Supported Sites](https://img.shields.io/badge/-Supported_Sites-brightgreen.svg?style=for-the-badge)](supportedsites.md) [![Discord](https://img.shields.io/discord/807245652072857610?color=blue&label=​&logo=discord&style=for-the-badge)](https://discord.gg/H5MNcFW63r) [![Doc Status](https://readthedocs.org/projects/yt-dlp/badge/?version=latest&style=for-the-badge)](https://yt-dlp.readthedocs.io) @@ -88,9 +88,9 @@ # NEW FEATURES * **Aria2c with HLS/DASH**: You can use `aria2c` as the external downloader for DASH(mpd) and HLS(m3u8) formats -* **New extractors**: AnimeLab, Philo MSO, Spectrum MSO, SlingTV MSO, Rcs, Gedi, bitwave.tv, mildom, audius, zee5, mtv.it, wimtv, pluto.tv, niconico users, discoveryplus.in, mediathek, NFHSNetwork, nebula, ukcolumn, whowatch, MxplayerShow, parlview (au), YoutubeWebArchive, fancode, Saitosan, ShemarooMe, telemundo, VootSeries, SonyLIVSeries, HotstarSeries, VidioPremier, VidioLive, RCTIPlus, TBS Live, douyin, pornflip, ParamountPlusSeries, ScienceChannel, Utreon, OpenRec +* **New extractors**: AnimeLab, Philo MSO, Spectrum MSO, SlingTV MSO, Cablevision MSO, Rcs, Gedi, bitwave.tv, mildom, audius, zee5, mtv.it, wimtv, pluto.tv, niconico users, discoveryplus.in, mediathek, NFHSNetwork, nebula, ukcolumn, whowatch, MxplayerShow, parlview (au), YoutubeWebArchive, fancode, Saitosan, ShemarooMe, telemundo, VootSeries, SonyLIVSeries, HotstarSeries, VidioPremier, VidioLive, RCTIPlus, TBS Live, douyin, pornflip, ParamountPlusSeries, ScienceChannel, Utreon, OpenRec, BandcampMusic, blackboardcollaborate, eroprofile albums, mirrativ -* **Fixed/improved extractors**: archive.org, roosterteeth.com, skyit, instagram, itv, SouthparkDe, spreaker, Vlive, akamai, ina, rumble, tennistv, amcnetworks, la7 podcasts, linuxacadamy, nitter, twitcasting, viu, crackle, curiositystream, mediasite, rmcdecouverte, sonyliv, tubi, tenplay, patreon, videa, yahoo, BravoTV, crunchyroll playlist, RTP, viki, Hotstar, vidio, vimeo, mediaset, Mxplayer +* **Fixed/improved extractors**: archive.org, roosterteeth.com, skyit, instagram, itv, SouthparkDe, spreaker, Vlive, akamai, ina, rumble, tennistv, amcnetworks, la7 podcasts, linuxacadamy, nitter, twitcasting, viu, crackle, curiositystream, mediasite, rmcdecouverte, sonyliv, tubi, tenplay, patreon, videa, yahoo, BravoTV, crunchyroll playlist, RTP, viki, Hotstar, vidio, vimeo, mediaset, Mxplayer, nbcolympics, ParamountPlus, Newgrounds, * **Subtitle extraction from manifests**: Subtitles can be extracted from streaming media manifests. See [commit/be6202f](https://github.com/yt-dlp/yt-dlp/commit/be6202f12b97858b9d716e608394b51065d0419f) for details @@ -98,11 +98,11 @@ # NEW FEATURES * **Portable Configuration**: Configuration files are automatically loaded from the home and root directories. See [configuration](#configuration) for details -* **Output template improvements**: Output templates can now have date-time formatting, numeric offsets, object traversal etc. See [output template](#output-template) for details. Even more advanced operations can also be done with the help of `--parse-metadata` +* **Output template improvements**: Output templates can now have date-time formatting, numeric offsets, object traversal etc. See [output template](#output-template) for details. Even more advanced operations can also be done with the help of `--parse-metadata` and `--replace-in-metadata` -* **Other new options**: `--sleep-requests`, `--convert-thumbnails`, `--write-link`, `--force-download-archive`, `--force-overwrites`, `--break-on-reject` etc +* **Other new options**: `--print`, `--sleep-requests`, `--convert-thumbnails`, `--write-link`, `--force-download-archive`, `--force-overwrites`, `--break-on-reject` etc -* **Improvements**: Multiple `--postprocessor-args` and `--downloader-args`, faster archive checking, more [format selection options](#format-selection) etc +* **Improvements**: Regex and other operators in `--match-filter`, multiple `--postprocessor-args` and `--downloader-args`, faster archive checking, more [format selection options](#format-selection) etc * **Plugin extractors**: Extractors can be loaded from an external file. See [plugins](#plugins) for details @@ -1339,7 +1339,7 @@ # MODIFYING METADATA The metadata obtained the the extractors can be modified by using `--parse-metadata` and `--replace-in-metadata` -`--replace-in-metadata FIELDS REGEX REPLACE` is used to replace text in any metatdata field using [python regular expression](https://docs.python.org/3/library/re.html#regular-expression-syntax). [Backreferences](https://docs.python.org/3/library/re.html?highlight=backreferences#re.sub) can be used in the replace string for advanced use. +`--replace-in-metadata FIELDS REGEX REPLACE` is used to replace text in any metadata field using [python regular expression](https://docs.python.org/3/library/re.html#regular-expression-syntax). [Backreferences](https://docs.python.org/3/library/re.html?highlight=backreferences#re.sub) can be used in the replace string for advanced use. The general syntax of `--parse-metadata FROM:TO` is to give the name of a field or a template (with same syntax as [output template](#output-template)) to extract data from, and the format to interpret it as, separated by a colon `:`. Either a [python regular expression](https://docs.python.org/3/library/re.html#regular-expression-syntax) with named capture groups or a similar syntax to the [output template](#output-template) (only `%(field)s` formatting is supported) can be used for `TO`. The option can be used multiple times to parse and modify various fields. diff --git a/supportedsites.md b/supportedsites.md index 6ec9105ad5..7e19b324c4 100644 --- a/supportedsites.md +++ b/supportedsites.md @@ -95,6 +95,7 @@ # Supported sites - **Bandcamp** - **Bandcamp:album** - **Bandcamp:weekly** + - **BandcampMusic** - **bangumi.bilibili.com**: BiliBili番剧 - **bbc**: BBC - **bbc.co.uk**: BBC iPlayer @@ -129,6 +130,7 @@ # Supported sites - **BitChuteChannel** - **bitwave:replay** - **bitwave:stream** + - **BlackboardCollaborate** - **BleacherReport** - **BleacherReportCMS** - **Bloomberg** @@ -295,6 +297,7 @@ # Supported sites - **Engadget** - **Eporner** - **EroProfile** + - **EroProfile:album** - **Escapist** - **ESPN** - **ESPNArticle** @@ -552,6 +555,8 @@ # Supported sites - **MinistryGrid** - **Minoto** - **miomio.tv** + - **mirrativ** + - **mirrativ:user** - **MiTele**: mitele.es - **mixcloud** - **mixcloud:playlist** @@ -703,6 +708,8 @@ # Supported sites - **OnionStudios** - **Ooyala** - **OoyalaExternal** + - **openrec** + - **openrec:capture** - **OraTV** - **orf:burgenland**: Radio Burgenland - **orf:fm4**: radio FM4 @@ -728,6 +735,7 @@ # Supported sites - **PalcoMP3:video** - **pandora.tv**: 판도라TV - **ParamountNetwork** + - **ParamountPlus** - **ParamountPlusSeries** - **parliamentlive.tv**: UK parliament videos - **Parlview** @@ -816,6 +824,7 @@ # Supported sites - **RCSVarious** - **RCTIPlus** - **RCTIPlusSeries** + - **RCTIPlusTV** - **RDS**: RDS.ca - **RedBull** - **RedBullEmbed**