Package org.schabi.newpipe.extractor.stream

Class StreamExtractor

java.lang.Object

org.schabi.newpipe.extractor.Extractor

org.schabi.newpipe.extractor.stream.StreamExtractor

Direct Known Subclasses:

BandcampStreamExtractor, MediaCCCLiveStreamExtractor, MediaCCCStreamExtractor, PeertubeStreamExtractor, SoundcloudStreamExtractor, YoutubeStreamExtractor

public abstract class StreamExtractor
extends Extractor

Scrapes information from a video/audio streaming service (eg, YouTube).

Nested Class Summary

Nested Classes

Modifier and Type	Class	Description
static class	StreamExtractor.Privacy

Field Summary

Fields

Modifier and Type	Field	Description
static int	NO_AGE_LIMIT
static long	UNKNOWN_SUBSCRIBER_COUNT

Constructor Summary

Constructors

Constructor	Description
StreamExtractor(StreamingService service, LinkHandler linkHandler)

Method Summary

Modifier and Type	Method	Description
int	getAgeLimit()	Get the age limit.
abstract java.util.List<AudioStream>	getAudioStreams()	This should return a list of available AudioStreams.
java.lang.String	getCategory()	The name of the category of the stream.
java.lang.String	getDashMpdUrl()	Get the dash mpd url.
Description	getDescription()	This is the stream description.
long	getDislikeCount()	The amount of dislikes a video/audio stream got.
java.lang.String	getErrorMessage()	Should analyse the webpage's document and extracts any error message there might be.
java.util.List<Frameset>	getFrames()	Should return a list of Frameset object that contains preview of stream frames
java.lang.String	getHlsUrl()	I am not sure if this is in use, and how this is used.
java.lang.String	getHost()	The host of the stream (Eg.
java.util.Locale	getLanguageInfo()	The locale language of the stream.
long	getLength()	This should return the length of a video in seconds.
java.lang.String	getLicence()	The name of the licence of the stream.
long	getLikeCount()	The amount of likes a video/audio stream got.
java.util.List<MetaInfo>	getMetaInfo()	Meta information about the stream.
StreamExtractor.Privacy	getPrivacy()	The privacy of the stream (Eg.
InfoItemsCollector<? extends InfoItem,? extends InfoItemExtractor>	getRelatedItems()	Should return a list of streams related to the current handled.
StreamInfoItemsCollector	getRelatedStreams()	Deprecated. Use getRelatedItems().
java.util.List<StreamSegment>	getStreamSegments()	The list of stream segments by timestamps for the stream.
abstract StreamType	getStreamType()	Get the StreamType.
java.util.List<Image>	getSubChannelAvatars()	The avatars of the sub-channel of the stream.
java.lang.String	getSubChannelName()	The name of the sub-channel of the stream.
java.lang.String	getSubChannelUrl()	The Url to the page of the sub-channel of the stream.
java.util.List<SubtitlesStream>	getSubtitles(MediaFormat format)	This will return a list of available SubtitlesStreams given by a specific type.
java.util.List<SubtitlesStream>	getSubtitlesDefault()	This will return a list of available SubtitlesStreams.
java.lang.String	getSupportInfo()	The support information of the stream.
java.util.List<java.lang.String>	getTags()	The list of tags of the stream.
java.lang.String	getTextualUploadDate()	The original textual date provided by the service.
abstract java.util.List<Image>	getThumbnails()	This will return the thumbnails of the stream.
long	getTimeStamp()	If the url you are currently handling contains a time stamp/seek, you can return the position it represents here.
protected long	getTimestampSeconds(java.lang.String regexPattern)	Override this function if the format of timestamp in the url is not the same format as that from youtube.
DateWrapper	getUploadDate()	A more general Calendar instance set to the date provided by the service. Implementations usually will just parse the date returned from the getTextualUploadDate().
java.util.List<Image>	getUploaderAvatars()	The image files/profile pictures/avatars of the creator/uploader of the stream.
abstract java.lang.String	getUploaderName()	The name of the creator/uploader of the stream.
long	getUploaderSubscriberCount()	The subscriber count of the uploader.
abstract java.lang.String	getUploaderUrl()	The Url to the page of the creator/uploader of the stream.
abstract java.util.List<VideoStream>	getVideoOnlyStreams()	This should return a list of available VideoStreams.
abstract java.util.List<VideoStream>	getVideoStreams()	This should return a list of available VideoStreams.
long	getViewCount()	The count of how many people have watched the video/listened to the audio stream.
boolean	isShortFormContent()	Whether the stream is a short-form content.
boolean	isUploaderVerified()	Whether the uploader has been verified by the service's provider.

Methods inherited from class org.schabi.newpipe.extractor.Extractor

assertPageFetched, fetchPage, forceContentCountry, forceLocalization, getBaseUrl, getDownloader, getExtractorContentCountry, getExtractorLocalization, getId, getLinkHandler, getName, getOriginalUrl, getService, getServiceId, getTimeAgoParser, getUrl, isPageFetched, onFetchPage

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Field Detail

NO_AGE_LIMIT

public static final int NO_AGE_LIMIT

See Also:

Constant Field Values

UNKNOWN_SUBSCRIBER_COUNT

public static final long UNKNOWN_SUBSCRIBER_COUNT

See Also:

Constant Field Values

Constructor Detail

StreamExtractor

public StreamExtractor(StreamingService service,
                       LinkHandler linkHandler)

Method Detail

getTextualUploadDate

@Nullable
public java.lang.String getTextualUploadDate()
                                      throws ParsingException

The original textual date provided by the service. Should be used as a fallback if getUploadDate() isn't provided by the service, or it fails for some reason.

If the stream is a live stream, null should be returned.

Returns:

The original textual date provided by the service, or null.

Throws:

ParsingException - if there is an error in the extraction

See Also:

getUploadDate()

getUploadDate

@Nullable
public DateWrapper getUploadDate()
                          throws ParsingException

A more general Calendar instance set to the date provided by the service.
Implementations usually will just parse the date returned from the getTextualUploadDate().

If the stream is a live stream, null should be returned.

Returns:

The date this item was uploaded, or null.

Throws:

ParsingException - if there is an error in the extraction or the extracted date couldn't be parsed.

See Also:

getTextualUploadDate()

getThumbnails

@Nonnull
public abstract java.util.List<Image> getThumbnails()
                                             throws ParsingException

This will return the thumbnails of the stream.

Returns:

the thumbnails of the stream

Throws:

ParsingException

getDescription

@Nonnull
public Description getDescription()
                           throws ParsingException

This is the stream description.

Returns:

The description of the stream/video or Description.EMPTY_DESCRIPTION if the description is empty.

Throws:

ParsingException

getAgeLimit

public int getAgeLimit()
                throws ParsingException

Get the age limit.

Returns:

The age which limits the content or 0 if there is no limit

Throws:

ParsingException - if an error occurs while parsing

getLength

public long getLength()
               throws ParsingException

This should return the length of a video in seconds.

Returns:

The length of the stream in seconds or 0 when it has no length (e.g. a livestream).

Throws:

ParsingException

getTimeStamp

public long getTimeStamp()
                  throws ParsingException

If the url you are currently handling contains a time stamp/seek, you can return the position it represents here. If the url has no time stamp simply return zero.

Returns:

the timestamp in seconds or 0 when there is no timestamp

Throws:

ParsingException

getViewCount

public long getViewCount()
                  throws ParsingException

The count of how many people have watched the video/listened to the audio stream. If the current stream has no view count or its not available simply return -1

Returns:

amount of views or -1 if not available.

Throws:

ParsingException

getLikeCount

public long getLikeCount()
                  throws ParsingException

The amount of likes a video/audio stream got. If the current stream has no likes or its not available simply return -1

Returns:

the amount of likes the stream got or -1 if not available.

Throws:

ParsingException

getDislikeCount

public long getDislikeCount()
                     throws ParsingException

The amount of dislikes a video/audio stream got. If the current stream has no dislikes or its not available simply return -1

Returns:

the amount of likes the stream got or -1 if not available.

Throws:

ParsingException

getUploaderUrl

@Nonnull
public abstract java.lang.String getUploaderUrl()
                                         throws ParsingException

The Url to the page of the creator/uploader of the stream. This must not be a homepage, but the page offered by the service the extractor handles. This url will be handled by the ChannelExtractor, so be sure to implement that one before you return a value here, otherwise NewPipe will crash if one selects this url.

Returns:

the url to the page of the creator/uploader of the stream or an empty string

Throws:

ParsingException

getUploaderName

@Nonnull
public abstract java.lang.String getUploaderName()
                                          throws ParsingException

The name of the creator/uploader of the stream. If the name is not available you can simply return an empty string.

Returns:

the name of the creator/uploader of the stream or an empty tring

Throws:

ParsingException

isUploaderVerified

public boolean isUploaderVerified()
                           throws ParsingException

Whether the uploader has been verified by the service's provider. If there is no verification implemented, return false.

Returns:

whether the uploader has been verified by the service's provider

Throws:

ParsingException

getUploaderSubscriberCount

public long getUploaderSubscriberCount()
                                throws ParsingException

The subscriber count of the uploader. If the subscriber count is not implemented, or is unavailable, return -1.

Returns:

the subscriber count of the uploader or -1L if not available

Throws:

ParsingException

getUploaderAvatars

@Nonnull
public java.util.List<Image> getUploaderAvatars()
                                         throws ParsingException

The image files/profile pictures/avatars of the creator/uploader of the stream.

If they are not available in the stream on specific cases, you must return an empty list for these ones, like it is made by default.

Returns:

the avatars of the sub-channel of the stream or an empty list (default)

Throws:

ParsingException

getSubChannelUrl

@Nonnull
public java.lang.String getSubChannelUrl()
                                  throws ParsingException

The Url to the page of the sub-channel of the stream. This must not be a homepage, but the page offered by the service the extractor handles. This url will be handled by the ChannelExtractor, so be sure to implement that one before you return a value here, otherwise NewPipe will crash if one selects this url.

Returns:

the url to the page of the sub-channel of the stream or an empty String

Throws:

ParsingException

getSubChannelName

@Nonnull
public java.lang.String getSubChannelName()
                                   throws ParsingException

The name of the sub-channel of the stream. If the name is not available you can simply return an empty string.

Returns:

the name of the sub-channel of the stream or an empty String

Throws:

ParsingException

getSubChannelAvatars

@Nonnull
public java.util.List<Image> getSubChannelAvatars()
                                           throws ParsingException

The avatars of the sub-channel of the stream.

If they are not available in the stream on specific cases, you must return an empty list for these ones, like it is made by default.

If the concept of sub-channels doesn't apply to the stream's service, keep the default implementation.

Returns:

the avatars of the sub-channel of the stream or an empty list (default)

Throws:

ParsingException

getDashMpdUrl

@Nonnull
public java.lang.String getDashMpdUrl()
                               throws ParsingException

Get the dash mpd url. If you don't know what a dash MPD is you can read about it here.

Returns:

the url as a string or an empty string or an empty string if not available

Throws:

ParsingException - if an error occurs while reading

getHlsUrl

@Nonnull
public java.lang.String getHlsUrl()
                           throws ParsingException

I am not sure if this is in use, and how this is used. However the frontend is missing support for HLS streams. Prove me if I am wrong. Please open an issue, or fix this description if you know whats up with this.

Returns:

The Url to the hls stream or an empty string if not available.

Throws:

ParsingException

getAudioStreams

public abstract java.util.List<AudioStream> getAudioStreams()
                                                     throws java.io.IOException,
                                                            ExtractionException

This should return a list of available AudioStreams. You can also return null or an empty list, however be aware that if you don't return anything in getVideoStreams(), getVideoOnlyStreams() and getDashMpdUrl() either the Collector will handle this as a failed extraction procedure.

Returns:

a list of audio only streams in the format of AudioStream

Throws:

java.io.IOException

ExtractionException

getVideoStreams

public abstract java.util.List<VideoStream> getVideoStreams()
                                                     throws java.io.IOException,
                                                            ExtractionException

This should return a list of available VideoStreams. Be aware this is the list of video streams which do contain an audio stream. You can also return null or an empty list, however be aware that if you don't return anything in getAudioStreams(), getVideoOnlyStreams() and getDashMpdUrl() either the Collector will handle this as a failed extraction procedure.

Returns:

a list of combined video and streams in the format of AudioStream

Throws:

java.io.IOException

ExtractionException

getVideoOnlyStreams

public abstract java.util.List<VideoStream> getVideoOnlyStreams()
                                                         throws java.io.IOException,
                                                                ExtractionException

This should return a list of available VideoStreams. Be aware this is the list of video streams which do NOT contain an audio stream. You can also return null or an empty list, however be aware that if you don't return anything in getAudioStreams(), getVideoStreams() and getDashMpdUrl() either the Collector will handle this as a failed extraction procedure.

Returns:

a list of video and streams in the format of AudioStream

Throws:

java.io.IOException

ExtractionException

getSubtitlesDefault

@Nonnull
public java.util.List<SubtitlesStream> getSubtitlesDefault()
                                                    throws java.io.IOException,
                                                           ExtractionException

This will return a list of available SubtitlesStreams. If no subtitles are available an empty list can be returned.

Returns:

a list of available subtitles or an empty list

Throws:

java.io.IOException

ExtractionException

getSubtitles

@Nonnull
public java.util.List<SubtitlesStream> getSubtitles(MediaFormat format)
                                             throws java.io.IOException,
                                                    ExtractionException

This will return a list of available SubtitlesStreams given by a specific type. If no subtitles in that specific format are available an empty list can be returned.

Parameters:

format - the media format by which the subtitles should be filtered

Returns:

a list of available subtitles or an empty list

Throws:

java.io.IOException

ExtractionException

getStreamType

public abstract StreamType getStreamType()
                                  throws ParsingException

Get the StreamType.

Returns:

the type of the stream

Throws:

ParsingException

getRelatedItems

@Nullable
public InfoItemsCollector<? extends InfoItem,? extends InfoItemExtractor> getRelatedItems()
                                                                                         throws java.io.IOException,
                                                                                                ExtractionException

Should return a list of streams related to the current handled. Many services show suggested streams. If you don't like suggested streams you should implement them anyway since they can be disabled by the user later in the frontend. The first related stream might be what was previously known as a next stream. If related streams aren't available simply return null.

Returns:

a list of InfoItems showing the related videos/streams

Throws:

java.io.IOException

ExtractionException

getRelatedStreams

@Deprecated
@Nullable
public StreamInfoItemsCollector getRelatedStreams()
                                           throws java.io.IOException,
                                                  ExtractionException

Deprecated.

Use getRelatedItems(). May be removed in a future version.

Returns:

The result of getRelatedItems() if it is a StreamInfoItemsCollector, null otherwise

Throws:

java.io.IOException

ExtractionException

getFrames

@Nonnull
public java.util.List<Frameset> getFrames()
                                   throws ExtractionException

Should return a list of Frameset object that contains preview of stream frames

Returns:

list of preview frames or empty list if frames preview is not supported or not found for specified stream

Throws:

ExtractionException

getErrorMessage

public java.lang.String getErrorMessage()

Should analyse the webpage's document and extracts any error message there might be.

Returns:

Error message; null if there is no error message.

getTimestampSeconds

protected long getTimestampSeconds(java.lang.String regexPattern)
                            throws ParsingException

Override this function if the format of timestamp in the url is not the same format as that from youtube.

Returns:

the time stamp/seek for the video in seconds

Throws:

ParsingException

getHost

@Nonnull
public java.lang.String getHost()
                         throws ParsingException

The host of the stream (Eg. peertube.cpy.re). If the host is not available, or if the service doesn't use a federated system, but a centralised system, you can simply return an empty string.

Returns:

the host of the stream or an empty string.

Throws:

ParsingException

getPrivacy

public StreamExtractor.Privacy getPrivacy()
                                   throws ParsingException

The privacy of the stream (Eg. Public, Private, Unlisted…).

Returns:

the privacy of the stream.

Throws:

ParsingException

getCategory

@Nonnull
public java.lang.String getCategory()
                             throws ParsingException

The name of the category of the stream. If the category is not available you can simply return an empty string.

Returns:

the category of the stream or an empty string.

Throws:

ParsingException

getLicence

@Nonnull
public java.lang.String getLicence()
                            throws ParsingException

The name of the licence of the stream. If the licence is not available you can simply return an empty string.

Returns:

the licence of the stream or an empty String.

Throws:

ParsingException

getLanguageInfo

@Nullable
public java.util.Locale getLanguageInfo()
                                 throws ParsingException

The locale language of the stream. If the language is not available you can simply return null. If the language is provided by a language code, you can return new Locale(language_code);

Returns:

the locale language of the stream or null.

Throws:

ParsingException

getTags

@Nonnull
public java.util.List<java.lang.String> getTags()
                                         throws ParsingException

The list of tags of the stream. If the tag list is not available you can simply return an empty list.

Returns:

the list of tags of the stream or Collections.emptyList().

Throws:

ParsingException

getSupportInfo

@Nonnull
public java.lang.String getSupportInfo()
                                throws ParsingException

The support information of the stream. see: https://framatube.org/videos/watch/ee408ec8-07cd-4e35-b884-fb681a4b9d37 (support button). If the support information are not available, you can simply return an empty String.

Returns:

the support information of the stream or an empty string.

Throws:

ParsingException

getStreamSegments

@Nonnull
public java.util.List<StreamSegment> getStreamSegments()
                                                throws ParsingException

The list of stream segments by timestamps for the stream. If the segment list is not available you can simply return an empty list.

Returns:

The list of segments of the stream or an empty list.

Throws:

ParsingException

getMetaInfo

@Nonnull
public java.util.List<MetaInfo> getMetaInfo()
                                     throws ParsingException

Meta information about the stream.

This can be information about the stream creator (e.g. if the creator is a public broadcaster) or further information on the topic (e.g. hints that the video might contain conspiracy theories or contains information about a current health situation like the Covid-19 pandemic).

The meta information often contains links to external sources like Wikipedia or the WHO.

Returns:

The meta info of the stream or an empty list if not provided.

Throws:

ParsingException

isShortFormContent

public boolean isShortFormContent()
                           throws ParsingException

Whether the stream is a short-form content.

Short-form contents are contents in the style of TikTok, YouTube Shorts, or Instagram Reels videos.

Returns:

whether the stream is a short-form content

Throws:

ParsingException