xZero707/youtube-dl
1
0
Fork 0
mirror of https://github.com/ytdl-org/youtube-dl.git synced 2024-12-12 13:06:30 +01:00
Code Issues Packages Projects Releases Wiki Activity
16,857 Commits 10 Branches 1,120 Tags 254 MiB
5dcd630dca
Commit Graph

536 Commits

Author SHA1 Message Date
Sergey M․
0a840f584c Rename bypass geo restriction options 2017-02-19 05:10:08 +08:00
Sergey M․
0016b84e16 Add faked X-Forwarded-For to formats' HTTP headers 2017-02-19 05:10:08 +08:00
Sergey M․
773f291dcb Add experimental geo restriction bypass mechanism 
Based on faking X-Forwarded-For HTTP header
 2017-02-19 05:10:08 +08:00
Yen Chi Hsuan
a4a554a793
[generic] Try parsing JWPlayer embedded videos (closes #12030) 2017-02-16 23:44:03 +08:00
Remita Amine
08a00eef79 [extractor/common] skip m3u8 manifests protected with Adobe Flash Access 2017-02-10 17:00:09 +01:00
Sergey M․
242a14a1f6
[extractor/common] Fix audio only with audio group in m3u8 (closes #11995) 2017-02-07 00:22:16 +07:00
Sergey M․
2aec7256ae
[extractor/common] Speed-up media tags regex (closes #11979) 2017-02-06 00:20:30 +07:00
Mattias Wadman
45024183ae [infoq] Add audio only format if available (#11565) 
* [infoq] Add audio only format if available

Refactor cookie code into a function.
Renamed formats to http_video, http_audio, rtmp_video
Renamed extract functions to video instead of videos as they return
one or no video.

* [infoq] Rename to _extract_cookies as it more than one

* [infoq] Remove redundant determine_ext

* [infoq] Add comment about hardcoded URL

* [infoq] Use _hidden_inputs instead of messy regex

* [infoq] Probe if audio URL is valid

Make it possible to pass headers to _is_valid_url

* [infoq] Add audio only test
 2017-02-03 12:10:13 +08:00
Sergey M․
e228616c6e
[extractor/common] Fix initialization template (closes #11605, closes #11825) 2017-01-29 06:57:39 +07:00
Sergey M․
c58c2d63cb
[extractor/common] Document forgotten fragment base and path interfaces 2017-01-29 05:57:53 +07:00
Sergey M․
d04621daf4
[extractor/common] Fix duration per dash segment (closes #11868) 2017-01-29 05:36:53 +07:00
Remita Amine
c4251b9aaa [common] add possibility to customize akamai manifest host 2017-01-13 10:21:36 +01:00
Yen Chi Hsuan
4606c34e19 [extractor/common] Allow non-lang in subtitles' keys 
See 264e77c406
 2016-12-25 01:50:50 +08:00
Remita Amine
a07588369f [common] improve detection for video only formats and m3u8 manifest(fixes #11507) 2016-12-22 10:02:56 +01:00
Remita Amine
7fe1592073 [common] fix dash codec information for mixed videos and fragment url construction(#11490) 2016-12-20 12:35:03 +01:00
Sergey M․
87a449c1ed
[extractor/common] Recognize DASH formats in html5 media entries 2016-12-17 23:03:13 +07:00
Remita Amine
8821a718cf [common] recognize hls manifests that contain video only formats(#11394) 2016-12-10 17:22:15 +01:00
Sergey M․
4ea4c0bb22
[extractor/common] Fix Bandwidth substitution in media template (closes #11175) 2016-11-13 05:43:34 +07:00
Sergey M․
f076d7972c
[extractor/common] Improve thumbnail extraction from JSON-LD 2016-11-12 23:01:05 +07:00
Sergey M․
c4c9b8440c
[extractor/common] Tolerate malformed RESOLUTION attribute in m3u8 manifests (closes #11113) 2016-11-04 05:02:31 +07:00
Sergey M․
3365ea8929
[extractor/common] Remove unused code 2016-11-02 02:34:23 +07:00
Sergey M․
1616f9b452
[extractor/common] Fix typo 2016-11-02 02:30:25 +07:00
Sergey M․
02dc0a36b7
[utils] Introduce base_url 2016-11-02 02:30:18 +07:00
Remita Amine
b2758123c5 add Basic support for Smooth Streaming protocol(#8118) 2016-11-02 01:54:45 +07:00
Sergey M․
b82c33dd67
[extractor/common] Improve mpd base URL extraction (closes #10909, closes #11079) 2016-11-01 01:15:46 +07:00
Remita Amine
0384932e3d [extractor/common] try to extract non smil wowza mpd manifests 2016-10-19 14:57:12 +01:00
Remita Amine
edd6074cea [extractor/common] detect f4m audio only formats 2016-10-19 14:42:48 +01:00
Sergey M․
8d76bdf12b
[extractor/common] Mention podcast in series fields section 2016-10-16 18:37:17 +07:00
Yen Chi Hsuan
cea364f70c
[extractor/common] Support HTML media elements without child nodes 2016-10-12 01:40:28 +08:00
Yen Chi Hsuan
98763ee354
[extractor/common] Add id and title helpers for generic IEs 2016-10-07 19:20:53 +08:00
Remita Amine
e71a450956 [common] add hdcore sign to akamai f4m formats 2016-09-24 21:55:53 +01:00
Yen Chi Hsuan
5968d7d2fe
[extractor/common] Improved support for HTML5 subtitles 
Ref: #10625

In a strict sense, <track>s with kind=captions are not subtitles. [1]
openload misuses this attribute, and I guess there will be more
examples, so I add it to common.py.

Also allow extracting information for subtitles-only <video> or <audio>
tags, which is the case of openload.

[1] https://www.w3.org/TR/html5/embedded-content-0.html#attr-track-kind
 2016-09-24 14:20:42 +08:00
Sergey M․
30d0b549be
[extractor/common] Add manifest_url for hls and hds formats 2016-09-17 21:33:38 +07:00
Sergey M․
86f4d14f81
Refactor fragments interface and dash segments downloader 
- Eliminate segment_urls and initialization_url
+ Introduce manifest_url (manifest may contain unfragmented data in this case url will be used for direct media URL and manifest_url for manifest itself correspondingly)
* Rewrite dashsegments downloader to use fragments data
* Improve generic mpd extraction
 2016-09-17 20:35:22 +07:00
Sergey M․
b4c1d6e800
[extractor/common] Expose fragments interface for dashsegments formats 2016-09-17 18:31:18 +07:00
Sergey M․
a0d5077c8d
[extractor/common] Introduce fragments interface 2016-09-17 18:31:09 +07:00
Remita Amine
6ad0219556 [common] add helper method for Wowza Streaming Engine format extraction 2016-09-16 19:30:38 +01:00
Sergey M․
dcce092e0a
[extractor/common] Simplify _get_netrc_login_info and carry long lines 2016-09-15 22:35:12 +07:00
Sergey M․
32443dd346
[extractor/common] Update _get_login_info's comment 2016-09-15 22:34:29 +07:00
Sergey M․
2133565cec
[extractor/common] Simplify _get_login_info 2016-09-15 22:26:37 +07:00
Sergey M․
c849836854
[utils] Improve _hidden_inputs 2016-09-15 21:54:48 +07:00
Remita Amine
1b6712ab23 [adobepass] add specific options for adobe pass authentication 
- add --ap-username and --ap-password option to specify
TV provider username and password in the cmd line
- add --ap-retries option to limit the number of retries
- add --list-ap-msi-ids to list the supported TV Providers
 2016-09-13 22:16:01 +01:00
Remita Amine
0def758782 [internetvideoarchive] extract all formats 2016-09-04 11:45:29 +01:00
Christian Pointner
dedb177029
Fix parsing of HTML5 media elements 
This fixes an error in _parse_html5_media_entries in case
an audio or video tag directly uses a src attribute insted
of <source> elements in it's body.
 2016-09-03 16:09:35 +07:00
Sergey M․
e816c9d158
[extractor/common] Simplify _extract_m3u8_formats 2016-09-01 22:18:16 +07:00
Sergey M․
9250181f37
[extractor/common] Restore NAME usage from EXT-X-MEDIA tag for formats codes in _extract_m3u8_formats (Closes #10522) 2016-09-01 21:37:25 +07:00
Remita Amine
f39ffc5877 [common] extract formats from #EXT-X-MEDIA tags 2016-08-25 08:49:44 +01:00
Remita Amine
c7c43a93ba [common] add helper method to extract akamai m3u8 and f4m formats 2016-08-22 07:49:34 +01:00
Yen Chi Hsuan
ad120ae1c5
[extractor/common] Change the default m3u8 protocol in HTML5 
Helper functions should have consistent default values
 2016-08-22 02:26:07 +08:00
Yen Chi Hsuan
520251c093
[extractor/common] Recognize m3u8 manifests in HTML5 multimedia tags 2016-08-19 23:53:47 +08:00
Remita Amine
2118fdd1a9 [common] add separate method for getting netrc ligin info 2016-08-14 11:55:28 +01:00
Sergey M․
321b5e082a
[extractor/common] Respect default in _search_json_ld 2016-08-08 22:36:18 +07:00
Remita Amine
37768f9242 [common] correctly lower the preference of m3u8 master manifest format 2016-08-07 10:59:09 +01:00
Remita Amine
d16b3c6677 [common] extract partOfTVSeries info in json-ld 2016-08-06 18:58:38 +01:00
Remita Amine
c57244cdb1 [common] lower the preference of m3u8 master manifest format 2016-08-06 18:55:05 +01:00
Sergey M․
46933a15d6
[extractor/common] Support root JSON-LD lists (Closes #10203) 2016-08-05 23:14:32 +07:00
Remita Amine
8b40854529 [common] lower proto_preference of rtsp formats 
Most of the time the RtspFD fail to download videos but it report
success of the download with this output:
[mpv] 0 bytes
[download] 100% of 0.00B
 2016-08-04 09:39:06 +01:00
Sergey M․
b070564efb
[extractor/common] Support multiple properties in _og_search_property 2016-08-02 22:55:14 +07:00
Sergey M․
40090e8d51
[extractor/common] Improve is_suitable 
In order to fix breakage introduced by a3aa814b77
 2016-07-26 23:54:06 +07:00
Sergey M․
f094834857
[extractor/common] Add support for $ in SegmentTemplate in MPD manifests 2016-07-24 10:27:16 +07:00
Remita Amine
00f4764cb7 [common] extract vbr, abr and fps for Unified Streaming Platform m3u8 manifests 2016-07-13 15:58:24 +01:00
Yen Chi Hsuan
0b68de3cc1 Merge pull request #8876 from remitamine/html5_media 
[extractor/common] add helper method to extract html5 media entries
 2016-07-10 23:40:45 +08:00
Sergey M․
95b31e266b
[extractor/common] Add expected_type in json ld routines 2016-07-09 03:28:04 +07:00
Sergey M․
6b3a3098b5
[extractor/common] Extract more metadata for VideoObject in _json_ld 2016-07-09 03:27:11 +07:00
Sergey M․
2de624fdd5
[extractor/common] Introduce filesize metafield for thumbnails 2016-07-09 03:24:36 +07:00
Déstin Reed
3fee7f636c
[flipagram] Add extractor 2016-07-09 03:23:32 +07:00
Yen Chi Hsuan
38cce791c7
Rename --cn-verfication-proxy to --geo-verification-proxy 
And deprecate the former one

Since commit f138873900, this option is
not limited to China websites, so rename it.
 2016-07-03 23:29:56 +08:00
Sergey M․
bf8dd79045
[extractor/common] Fix sorting with custom field preference 2016-06-26 21:09:07 +07:00
remitamine
59bbe4911a [extractor/common] add helper method to extract html5 media entries 2016-06-26 14:04:08 +01:00
Sergey M․
88d9f6c0c4
[utils] Add support for name list in _html_search_meta 2016-06-26 16:57:14 +07:00
remitamine
e154c65128 [downloader/hls] Add support for AES-128 encrypted segments in hlsnative downloader 2016-06-19 01:01:40 +01:00
Yen Chi Hsuan
a6571f1073
[common] Fix <bootstrapInfo> detection in F4M manifests 
Regression since 0a5685b26f
 2016-06-08 00:19:33 +08:00
Sergey M․
77b8b4e696
[extractor/common] Borrow quality metadata from parent set-level manifest for f4m 2016-05-27 01:47:44 +06:00
Yen Chi Hsuan
fac2af3c51
[common] Fix m3u8 extraction in f4m manifests 2016-05-27 01:41:27 +08:00
Yen Chi Hsuan
448bb5f333
[common] Fix non-bootstrapped support in f4m 2016-05-27 00:03:48 +08:00
Yen Chi Hsuan
240b60453e
[common] Support m3u8 in f4m manifests 
Related: #9531
 2016-05-26 21:55:43 +08:00
Yen Chi Hsuan
0a5685b26f
[common] Support non-bootstraped streams in f4m manifests 
Related: #9531
 2016-05-26 21:41:47 +08:00
Yen Chi Hsuan
7b2fcbfd4e
[common] Skip TYPE=CLOSED-CAPTIONS lines in m3u8 manifests 
According to [1], valid values for TYPE are AUDIO, VIDEO, SUBTITLES
and CLOSED-CAPTIONS. Such a value is found in Anvato master playlists,
though I don't use _extract_m3u8_formats() in the end.

Part of #9522.

[1] https://tools.ietf.org/html/draft-pantos-http-live-streaming-19#section-4.3.4.1
 2016-05-21 13:16:28 +08:00
Yen Chi Hsuan
16da9bbc29
[common] Add _m3u8_meta_format() template 
For extractors who handle m3u8 manifests by themselves. (eg., AnvatoIE)

Part of #9522
 2016-05-21 13:15:28 +08:00
Yen Chi Hsuan
ad96b4c8f5
[common] Extract audio formats in SMIL 
Found in http://www.cbc.ca/player/play/2657631896

Closes #5156
 2016-05-20 19:02:53 +08:00
Sergey M․
ed56f26039
[extractor/common] Improve name extraction for m3u8 formats 2016-05-15 03:34:35 +06:00
Sergey M․
8a92e51c60
[extractor/common] Relax wording for creator metafield 2016-05-02 21:31:35 +06:00
Yen Chi Hsuan
e9c6cdf4a1
[common] Fix format_id construction for HLS 2016-04-29 22:50:16 +08:00
Kagami Hiiragi
b24d6336a7 [vlive] Add support for live videos 2016-04-29 14:22:50 +03:00
Yen Chi Hsuan
d6712378e7
Merge branch 'akamai_pv' of https://github.com/remitamine/youtube-dl into remitamine-akamai_pv 2016-04-25 21:02:02 +08:00
remitamine
fb72ec58ae [extractor/common] do not process f4m manifest that contain akamai playerVerificationChallenge 2016-04-25 13:37:03 +01:00
Yen Chi Hsuan
2c0d9c6217
[extractor/common] Allow empty post data 2016-04-21 13:06:06 +08:00
Sergey M․
49caf3307f
[extractor/common] Remove irrelevant comment 2016-04-10 17:10:27 +06:00
Sergey M․
bacec0397f [extractor/common] Relax _hidden_inputs 2016-04-08 23:33:45 +06:00
Sergey M․
fb38aa8b53 [extractor/common] Support arbitrary format strings for template based identifiers in mpd manifests (Closes #9119, closes #9120) 2016-04-08 22:48:08 +06:00
Sergey M․
7a93ab5f3f [extractor/common] Introduce music album metafields 2016-04-07 02:53:53 +06:00
Sergey M․
b507cc925b [extractor/common] Carry long line 2016-04-02 18:49:58 +06:00
Sergey M․
db8ee7ec05 [extractor/common] Fix numeric identifiers conversion in DASH URL templates 2016-04-02 18:48:05 +06:00
remitamine
df634be2ed [common] prefer using mime type over ext for smil subtitle extraction 
the subtitle ext for http://www.cnet.com/videos/download-amazon-prime-movies-and-tv/
is adb_xml while using the mime type it get tt(application/smptett+xml)
 2016-04-01 19:47:49 +01:00
Sergey M․
41d06b0424 [extractor/common] Improve _request_webpage 
* Do not ignore data, headers and query for Requests
* Default values for headers and query switched to dicts since these are used by urllib itself
 2016-03-31 22:58:38 +06:00
Sergey M․
b22ca76204 [extractor/common] Filter out unsupported encrypted media for f4m formats (Closes #8573) 2016-03-27 07:42:38 +06:00
Sergey M․
19dbaeece3 Remove _sort_formats from _extract_*_formats methods 
Now _sort_formats should be called explicitly.
_sort_formats has been added to all the necessary places in code.

Closes #8051
 2016-03-27 07:03:08 +06:00
Sergey M․
15707c7e02 [compat] Add compat_urllib_parse_urlencode and eliminate encode_dict 
encode_dict functionality has been improved and moved directly into compat_urllib_parse_urlencode
All occurrences of compat_urllib_parse.urlencode throughout the codebase have been replaced by compat_urllib_parse_urlencode

Closes #8974
 2016-03-26 01:46:57 +06:00
remitamine
49dea4913b Merge pull request #8513 from remitamine/dash-sort 
[extractor/common] fix dash formats sorting
 2016-03-15 18:39:50 +01:00
Sergey M․
0fdbb3322b [extractor/common] Add _parse_f4m_formats routine 2016-03-13 03:16:08 +06:00
remitamine
09f572fbc0 [extractor/common] add transform_source to _download_smil and _extract_smil_formats 2016-03-11 22:37:07 +01:00
remitamine
15bf934de5 Merge pull request #8819 from remitamine/simple-webpage-requests 
[extractor/common] simplify using data, headers and query params with _download_* methods
 2016-03-11 18:19:43 +01:00
remitamine
cdfee16818 [extractor/common] add data, headers and query params to _request_webpage 2016-03-11 18:12:50 +01:00
Yen Chi Hsuan
a6c8b75904 [common] Use mimeType to determine file extensions (#8766) 2016-03-11 23:51:42 +08:00
Yen Chi Hsuan
64f08d4ff2 Merge pull request #8766 from yan12125/dash-detect-ext 
Detect file extensions of DASH formats from their codecs
 2016-03-11 21:40:07 +08:00
Yen Chi Hsuan
af7d5a63b2 [common] Document protocol http_dash_segments 2016-03-06 17:47:07 +08:00
Yen Chi Hsuan
2def60c5f3 [common] Use codec2ext for DASH formats (#8764) 2016-03-05 18:18:39 +08:00
Yen Chi Hsuan
e9c0cdd389 [jython] Introduce compat_os_name 
os.name is always 'java' on Jython
 2016-03-03 19:24:24 +08:00
Sergey M․
7bcd2830dd [extractor/common] Document uploader_url 2016-03-02 23:31:24 +06:00
Sergey M․
2bc0c46f98 [extractor/common] Document license metafield 2016-03-02 23:06:39 +06:00
Sergey M․
d77ab8e255 Add --mark-watched feature (Closes #5054) 2016-03-01 01:01:33 +06:00
Sergey M․
9cdffeeb3f [extractor/common] Clarify rationale on media playlist detection 2016-02-27 07:01:11 +06:00
Sergey M․
fbb6edd298 [extractor/common] Properly extract audio only formats in master m3u8 playlists 2016-02-27 06:48:13 +06:00
Sergey M․
f5bdb44443 [extractor/common] Add _remove_duplicate_formats 2016-02-22 01:19:39 +06:00
remitamine
cafcf657a4 add more subtitles mime types to mimetype2ext and fix the platform subtitle extraction 2016-02-20 22:02:03 +01:00
Sergey M․
611c1dd96e [refactor] Single quotes consistency 2016-02-14 15:37:17 +06:00
Sergey M․
d800609c62 [refactor] Do not specify redundant None as second argument in dict.get() 2016-02-14 14:25:04 +06:00
Sergey M․
bb20526b64 [extractor/common] Improve base url construction 2016-02-13 00:13:56 +06:00
remitamine
c349456ef6 [extractor/common] strip http urls in smil manifest 2016-02-12 17:38:48 +01:00
remitamine
81e1c4e2fc [extractor/common] remove duplicate rtmp formats in smil manifest 2016-02-11 17:58:48 +01:00
remitamine
dd86780596 [extractor/common] fix dash formats sorting 2016-02-11 10:55:50 +01:00
remitamine
154c209e2d [extractor/common] improve dash format ids 2016-02-11 10:33:26 +01:00
remitamine
51e9094f4a [extractor/common] extract youtube dash formats filesize(fixes #8480) 2016-02-09 20:05:39 +01:00
remitamine
d413095f7e [extractor/common] remove duplicated formats and subtiles in smil manifests 2016-02-09 17:15:41 +01:00
remitamine
6a3828fddd [common] use float conversion instead of using division from __future__ 2016-02-06 14:27:04 +01:00
remitamine
91cb6b5065 rename _parse_mpd to _parse_mpd_formats and add default value for mpd namespace 2016-02-06 14:03:48 +01:00
remitamine
0826a0b555 [common] sort dash formats 2016-02-06 06:52:48 +01:00
remitamine
255732f0d3 [common] fix segment duration calculation 2016-02-03 23:57:08 +01:00
remitamine
53c269c6fd [common] fix media_template string formating 2016-02-03 23:54:34 +01:00
remitamine
675d001633 [common] skip drm protected dash formats 2016-02-03 18:44:43 +01:00
remitamine
d577c79632 [common] ignore ISO 639-2 generic codes 2016-02-03 13:24:07 +01:00
remitamine
f14be22816 [common] remove duplicate reference to namespace 2016-02-02 22:02:08 +01:00
remitamine
9c74423510 [common] fix media template regex 2016-02-02 18:30:31 +01:00
remitamine
1bac34556f [common] add a generic support for mpd manifests 2016-02-02 18:09:25 +01:00
Yen Chi Hsuan
2d2fa82d17 [common] Add _extract_dash_manifest_formats 2016-01-30 22:52:23 +08:00
Yen Chi Hsuan
c94678957f [common] Remove unused arguments 2016-01-30 22:45:16 +08:00
Yen Chi Hsuan
16f38a699f [common] Rename to namespace 
For consistency with _parse_smil_*
 2016-01-30 22:40:56 +08:00
Yen Chi Hsuan
df374b5222 [common] Prefer the manifest than formats_dict in determining codecs 2016-01-30 21:42:27 +08:00
Yen Chi Hsuan
5ea1eb78f5 [common] Fix for youtube 2016-01-30 21:36:01 +08:00
Yen Chi Hsuan
b323e1707d [common] Modify _parse_dash_manifest for use in Facebook 2016-01-30 21:27:43 +08:00
Yen Chi Hsuan
17b598d30c [common] _parse_dash_manifest() from youtube.py 2016-01-30 21:05:55 +08:00
Sergey M․
350cf045d8 [extractor/common] Restrict checks when auto calculating tbr 2016-01-30 01:47:46 +06:00
remitamine
a9d5f12fec Merge pull request #8328 from remitamine/hls-master-detect 
[extractor/common] detect media playlist in _extract_m3u8_formats
 2016-01-27 18:07:30 +01:00
remitamine
7f32e5dc35 [extractor/common] detect media playlist in _extract_m3u8_formats 2016-01-27 17:53:42 +01:00
Sergey M․
b0d21deda9 [extractor/common] Auto calculate tbr when missing 2016-01-27 21:11:17 +06:00
Yen Chi Hsuan
77f785076f [common] Keep full codec name from m3u8 manifests 
See #8293. This is for consistency between YouTube and HLS formats.
 2016-01-25 01:03:46 +08:00
Yen Chi Hsuan
0b26ba3fc8 [extractor/common] Allow passing more parameters to _search_json_ld 2016-01-16 20:45:36 +08:00
Sergey M․
4ca2a3cf3c [extractor/common] Add initial support for JSON-LD metadata extraction into info_dict 2016-01-16 00:36:02 +06:00
Jakub Wilk
dfb1b1468c Fix typos 
Closes #8200.
 2016-01-10 17:24:28 +01:00
Sergey M
3f3343cd3e Merge pull request #8061 from dstftw/introduce-chapter-and-series-fields 
Introduce chapter and series fields
 2016-01-03 03:54:22 +06:00