LesLibres.org/Rss-Bridge
2
0
Fork 0
Code Issues Pull requests Releases Wiki Activity

Merge tag '2019-06-08' into kt_bridge

Browse source
This commit is contained in:
Knah Tsaeb 2019-06-17 15:39:52 +02:00
commit a627aed0da
40 changed files with 1430 additions and 1005 deletions
Show stats Download patch file Download diff file Expand all files Collapse all files

11
.dockerignore
View file

@ -1,7 +1,14 @@
.git
.gitattributes
.github/*
.travis.yml
cache/*
CONTRIBUTING.md
DEBUG
Dockerfile
whitelist.txt
phpcompatibility.xml
phpcs.xml
CONTRIBUTING.md
phpcs.xml
scalingo.json
tests/*
whitelist.txt

57
.gitattributes vendored
View file

@ -10,27 +10,40 @@
*.dbproj merge=union
# Standard to msysgit
*.doc diff=astextplain
*.DOC diff=astextplain
*.docx diff=astextplain
*.DOCX diff=astextplain
*.dot diff=astextplain
*.DOT diff=astextplain
*.pdf diff=astextplain
*.PDF diff=astextplain
*.rtf diff=astextplain
*.RTF diff=astextplain
*.doc diff=astextplain
*.DOC diff=astextplain
*.docx diff=astextplain
*.DOCX diff=astextplain
*.dot diff=astextplain
*.DOT diff=astextplain
*.pdf diff=astextplain
*.PDF diff=astextplain
*.rtf diff=astextplain
*.RTF diff=astextplain
# Ignore files in git archive (i.e. GitHub release builds)
Dockerfile export-ignore
.travis.yml export-ignore
.github/ export-ignore
.gitattributes export-ignore
.gitignore export-ignore
.dockerignore export-ignore
scalingo.json export-ignore
phpunit.xml export-ignore
phpcs.xml export-ignore
phpcompatibility.xml export-ignore
tests/ export-ignore
cache/.gitkeep export-ignore
## Docker
Dockerfile export-ignore
.dockerignore export-ignore
## Travis
.travis.yml export-ignore
## GitHub
.github/ export-ignore
## Git
.gitattributes export-ignore
.gitignore export-ignore
## Scalingo
scalingo.json export-ignore
## RSS-Bridge
phpunit.xml export-ignore
phpcs.xml export-ignore
phpcompatibility.xml export-ignore
tests/ export-ignore
cache/.gitkeep export-ignore
bridges/DemoBridge.php export-ignore
bridges/FeedExpanderExampleBridge.php export-ignore
## Composer
composer.json export-ignore
composer.lock export-ignore
## Heroku
app.json export-ignore

5
.github/ISSUE_TEMPLATE/bridge-request-template.md → .github/ISSUE_TEMPLATE/bridge-request.md vendored
View file

@ -1,6 +1,9 @@
---
name: Bridge request template
name: Bridge request
about: Use this template for requesting a new bridge
title: Bridge request for ...
labels: Bridge-Request
assignees: ''
---

38
.github/ISSUE_TEMPLATE/bug_report.md vendored Normal file
View file

@ -0,0 +1,38 @@
---
name: Bug report
about: Create a report to help us improve
title: ''
labels: Bug-Report
assignees: ''
---
**Describe the bug**
A clear and concise description of what the bug is.
**To Reproduce**
Steps to reproduce the behavior:
1. Go to '...'
2. Click on '....'
3. Scroll down to '....'
4. See error
**Expected behavior**
A clear and concise description of what you expected to happen.
**Screenshots**
If applicable, add screenshots to help explain your problem.
**Desktop (please complete the following information):**
- OS: [e.g. iOS]
- Browser [e.g. chrome, safari]
- Version [e.g. 22]
**Smartphone (please complete the following information):**
- Device: [e.g. iPhone6]
- OS: [e.g. iOS8.1]
- Browser [e.g. stock browser, safari]
- Version [e.g. 22]
**Additional context**
Add any other context about the problem here.

20
.github/ISSUE_TEMPLATE/feature_request.md vendored Normal file
View file

@ -0,0 +1,20 @@
---
name: Feature request
about: Suggest an idea for this project
title: ''
labels: Feature-Request
assignees: ''
---
**Is your feature request related to a problem? Please describe.**
A clear and concise description of what the problem is. Ex. I'm always frustrated when [...]
**Describe the solution you'd like**
A clear and concise description of what you want to happen.
**Describe alternatives you've considered**
A clear and concise description of any alternative solutions or features you've considered.
**Additional context**
Add any other context or screenshots about the feature request here.

3
.gitignore vendored
View file

@ -240,3 +240,6 @@ config.ini.php
#Auth
.htaccess
.htpasswd
#Crawler
robots.txt

12
Dockerfile
View file

@ -1,5 +1,11 @@
FROM ulsmith/alpine-apache-php7
FROM php:7-apache
COPY ./ /app/public/
ENV APACHE_DOCUMENT_ROOT=/app
RUN chown -R apache:root /app/public
RUN mv "$PHP_INI_DIR/php.ini-production" "$PHP_INI_DIR/php.ini" \
&& apt-get --yes update && apt-get --yes install libxml2-dev \
&& docker-php-ext-install -j$(nproc) simplexml \
&& sed -ri -e 's!/var/www/html!${APACHE_DOCUMENT_ROOT}!g' /etc/apache2/sites-available/*.conf \
&& sed -ri -e 's!/var/www/!${APACHE_DOCUMENT_ROOT}!g' /etc/apache2/apache2.conf /etc/apache2/conf-available/*.conf
COPY --chown=www-data:www-data ./ /app/

9
README.md
View file

@ -85,7 +85,7 @@ Deploy
Thanks to the community, hosting your own instance of RSS-Bridge is as easy as clicking a button!
[![Deploy on Scalingo](https://cdn.scalingo.com/deploy/button.svg)](https://my.scalingo.com/deploy?source=https://github.com/sebsauvage/rss-bridge)
[![Deploy to Docker Cloud](https://files.cloud.docker.com/images/deploy-to-dockercloud.svg)](https://cloud.docker.com/stack/deploy/?repo=https://github.com/rss-bridge/rss-bridge)
[![Deploy to Heroku](https://www.herokucdn.com/deploy/button.svg)](https://heroku.com/deploy)
Getting involved
===
@ -116,13 +116,14 @@ https://gist.github.com/LogMANOriginal/da00cd1e5f0ca31cef8e193509b17fd8
* [Ahiles3005](https://github.com/Ahiles3005)
* [Albirew](https://github.com/Albirew)
* [aledeg](https://github.com/aledeg)
* [alex73](https://github.com/alex73)
* [alexAubin](https://github.com/alexAubin)
* [AmauryCarrade](https://github.com/AmauryCarrade)
* [AntoineTurmel](https://github.com/AntoineTurmel)
* [ArthurHoaro](https://github.com/ArthurHoaro)
* [Astalaseven](https://github.com/Astalaseven)
* [Astyan-42](https://github.com/Astyan-42)
* [az5he6ch](https://github.com/az5he6ch)
* [azdkj532](https://github.com/azdkj532)
* [b1nj](https://github.com/b1nj)
* [benasse](https://github.com/benasse)
* [captn3m0](https://github.com/captn3m0)
@ -156,6 +157,7 @@ https://gist.github.com/LogMANOriginal/da00cd1e5f0ca31cef8e193509b17fd8
* [jdigilio](https://github.com/jdigilio)
* [JeremyRand](https://github.com/JeremyRand)
* [Jocker666z](https://github.com/Jocker666z)
* [killruana](https://github.com/killruana)
* [klimplant](https://github.com/klimplant)
* [kranack](https://github.com/kranack)
* [kraoc](https://github.com/kraoc)
@ -173,7 +175,6 @@ https://gist.github.com/LogMANOriginal/da00cd1e5f0ca31cef8e193509b17fd8
* [mdemoss](https://github.com/mdemoss)
* [melangue](https://github.com/melangue)
* [metaMMA](https://github.com/metaMMA)
* [mickael-bertrand](https://github.com/mickael-bertrand)
* [mitsukarenai](https://github.com/mitsukarenai)
* [MonsieurPoutounours](https://github.com/MonsieurPoutounours)
* [mr-flibble](https://github.com/mr-flibble)
@ -208,8 +209,10 @@ https://gist.github.com/LogMANOriginal/da00cd1e5f0ca31cef8e193509b17fd8
* [thefranke](https://github.com/thefranke)
* [TheRadialActive](https://github.com/TheRadialActive)
* [triatic](https://github.com/triatic)
* [VerifiedJoseph](https://github.com/VerifiedJoseph)
* [WalterBarrett](https://github.com/WalterBarrett)
* [wtuuju](https://github.com/wtuuju)
* [xurxof](https://github.com/xurxof)
* [yardenac](https://github.com/yardenac)
* [ZeNairolf](https://github.com/ZeNairolf)

8
app.json Normal file
View file

@ -0,0 +1,8 @@
{
"service": "Heroku",
"name": "RSS-Bridge",
"description": "RSS-Bridge is a PHP project capable of generating RSS and Atom feeds for websites which don't have one.",
"repository": "https://github.com/RSS-Bridge/rss-bridge",
"keywords": ["php", "rss-bridge", "rss"]
}

3
bridges/Arte7Bridge.php
View file

@ -91,7 +91,8 @@ class Arte7Bridge extends BridgeAbstract {
'Authorization: Bearer ' . self::API_TOKEN
);
$input = getContents($url, $header) or die('Could not request ARTE.');
$input = getContents($url, $header)
or returnServerError('Could not request ARTE.');
$input_json = json_decode($input, true);
foreach($input_json['videos'] as $element) {

103
bridges/BinanceBridge.php Normal file
View file

@ -0,0 +1,103 @@
<?php
class BinanceBridge extends BridgeAbstract {
const NAME = 'Binance';
const URI = 'https://www.binance.com';
const DESCRIPTION = 'Subscribe to the Binance blog or the Binance Zendesk announcements.';
const MAINTAINER = 'thefranke';
const CACHE_TIMEOUT = 3600; // 1h
const PARAMETERS = array( array(
'category' => array(
'name' => 'category',
'type' => 'list',
'exampleValue' => 'Blog',
'title' => 'Select a category',
'values' => array(
'Blog' => 'Blog',
'Announcements' => 'Announcements'
)
)
));
public function getIcon() {
return 'https://bin.bnbstatic.com/static/images/common/favicon.ico';
}
public function getName() {
return self::NAME . ' ' . $this->getInput('category');
}
public function getURI() {
if ($this->getInput('category') == 'Blog')
return self::URI . '/en/blog';
else
return 'https://binance.zendesk.com/hc/en-us/categories/115000056351-Announcements';
}
protected function collectBlogData() {
$html = getSimpleHTMLDOM($this->getURI())
or returnServerError('Could not fetch Binance blog data.');
foreach($html->find('div[direction="row"]') as $element) {
$date = $element->find('div[direction="column"]', 0);
$day = $date->find('div', 0)->innertext;
$month = $date->find('div', 1)->innertext;
$extractedDate = $day . ' ' . $month;
$abstract = $element->find('div[direction="column"]', 1);
$a = $abstract->find('a', 0);
$uri = self::URI . $a->href;
$title = $a->innertext;
$full = getSimpleHTMLDOMCached($uri);
$content = $full->find('div.desc', 1);
$item = array();
$item['title'] = $title;
$item['uri'] = $uri;
$item['timestamp'] = strtotime($extractedDate);
$item['author'] = 'Binance';
$item['content'] = $content;
$this->items[] = $item;
if (count($this->items) >= 10)
break;
}
}
protected function collectAnnouncementData() {
$html = getSimpleHTMLDOM($this->getURI())
or returnServerError('Could not fetch Zendesk announcement data.');
foreach($html->find('a.article-list-link') as $a) {
$title = $a->innertext;
$uri = 'https://binance.zendesk.com' . $a->href;
$full = getSimpleHTMLDOMCached($uri);
$content = $full->find('div.article-body', 0);
$date = $full->find('time', 0)->getAttribute('datetime');
$item = array();
$item['title'] = $title;
$item['uri'] = $uri;
$item['timestamp'] = strtotime($date);
$item['author'] = 'Binance';
$item['content'] = $content;
$this->items[] = $item;
if (count($this->items) >= 10)
break;
}
}
public function collectData() {
if ($this->getInput('category') == 'Blog')
$this->collectBlogData();
else
$this->collectAnnouncementData();
}
}

142
bridges/BrutBridge.php Normal file
View file

@ -0,0 +1,142 @@
<?php
class BrutBridge extends BridgeAbstract {
const NAME = 'Brut Bridge';
const URI = 'https://www.brut.media';
const DESCRIPTION = 'Returns 5 newest videos by category and edition';
const MAINTAINER = 'VerifiedJoseph';
const PARAMETERS = array(array(
'category' => array(
'name' => 'Category',
'type' => 'list',
'values' => array(
'News' => 'news',
'International' => 'international',
'Economy' => 'economy',
'Science and Technology' => 'science-and-technology',
'Entertainment' => 'entertainment',
'Sports' => 'sport',
'Nature' => 'nature',
),
'defaultValue' => 'news',
),
'edition' => array(
'name' => ' Edition',
'type' => 'list',
'values' => array(
'United States' => 'us',
'United Kingdom' => 'uk',
'France' => 'fr',
'India' => 'in',
'Mexico' => 'mx',
),
'defaultValue' => 'us',
)
)
);
const CACHE_TIMEOUT = 1800; // 30 mins
private $videoId = '';
private $videoType = '';
private $videoImage = '';
public function collectData() {
$html = getSimpleHTMLDOM($this->getURI())
or returnServerError('Could not request: ' . $this->getURI());
$results = $html->find('div.results', 0);
foreach($results->find('li.col-6.col-sm-4.col-md-3.col-lg-2.px-2.pb-4') as $index => $li) {
$item = array();
$videoPath = self::URI . $li->children(0)->href;
$videoPageHtml = getSimpleHTMLDOMCached($videoPath, 3600)
or returnServerError('Could not request: ' . $videoPath);
$this->videoImage = $videoPageHtml->find('meta[name="twitter:image"]', 0)->content;
$this->processTwitterImage();
$description = $videoPageHtml->find('div.description', 0);
$item['uri'] = $videoPath;
$item['title'] = $description->find('h1', 0)->plaintext;
if ($description->find('div.date', 0)->children(0)) {
$description->find('div.date', 0)->children(0)->outertext = '';
}
$item['content'] = $this->processContent(
$description
);
$item['timestamp'] = $this->processDate($description);
$item['enclosures'][] = $this->videoImage;
$this->items[] = $item;
if (count($this->items) >= 5) {
break;
}
}
}
public function getURI() {
if (!is_null($this->getInput('edition')) && !is_null($this->getInput('category'))) {
return self::URI . '/' . $this->getInput('edition') . '/' . $this->getInput('category');
}
return parent::getURI();
}
private function processDate($description) {
if ($this->getInput('edition') === 'uk') {
$date = DateTime::createFromFormat('d/m/Y H:i', $description->find('div.date', 0)->innertext);
return strtotime($date->format('Y-m-d H:i:s'));
}
return strtotime($description->find('div.date', 0)->innertext);
}
private function processContent($description) {
$content = '<video controls poster="' . $this->videoImage . '" preload="none">
<source src="https://content.brut.media/video/' . $this->videoId . '-' . $this->videoType . '-web.mp4"
type="video/mp4">
</video>';
$content .= '<p>' . $description->find('h2.mb-1', 0)->innertext . '</p>';
if ($description->find('div.text.pb-3', 0)->children(1)->class != 'date') {
$content .= '<p>' . $description->find('div.text.pb-3', 0)->children(1)->innertext . '</p>';
}
return $content;
}
private function processTwitterImage() {
/**
* Extract video ID + type from twitter image
*
* Example (wrapped):
* https://img.brut.media/thumbnail/
* the-life-of-rita-moreno-2cce75b5-d448-44d2-a97c-ca50d6470dd4-square.jpg
* ?ts=1559337892
*/
$fpath = parse_url($this->videoImage, PHP_URL_PATH);
$fname = basename($fpath);
$fname = substr($fname, 0, strrpos($fname, '.'));
$parts = explode('-', $fname);
if (end($parts) === 'auto') {
$key = array_search('auto', $parts);
unset($parts[$key]);
}
$this->videoId = implode('-', array_splice($parts, -6, 5));
$this->videoType = end($parts);
}
}

14
bridges/FB2Bridge.php
View file

@ -72,15 +72,15 @@ class FB2Bridge extends BridgeAbstract {
$pageInfo = $this->getPageInfos($page, $cookies);
if($pageInfo['userId'] === null) {
echo <<<EOD
returnClientError(<<<EOD
Unable to get the page id. You should consider getting the ID by hand, then importing it into FB2Bridge
EOD;
die();
EOD
);
} elseif($pageInfo['userId'] == -1) {
echo <<<EOD
returnClientError(<<<EOD
This page is not accessible without being logged in.
EOD;
die();
EOD
);
}
}
@ -95,7 +95,7 @@ EOD;
foreach($html->find('article') as $content) {
$item = array();
//echo $content; die();
preg_match('/publish_time\\\":([0-9]+),/', $content->getAttribute('data-store', 0), $match);
if(isset($match[1]))
$timestamp = $match[1];

8
bridges/GOGBridge.php
View file

@ -8,8 +8,8 @@ class GOGBridge extends BridgeAbstract {
public function collectData() {
$values = getContents('https://www.gog.com/games/ajax/filtered?limit=25&sort=new') or
die('Unable to get the news pages from GOG !');
$values = getContents('https://www.gog.com/games/ajax/filtered?limit=25&sort=new')
or returnServerError('Unable to get the news pages from GOG !');
$decodedValues = json_decode($values);
$limit = 0;
@ -38,8 +38,8 @@ class GOGBridge extends BridgeAbstract {
private function buildGameContentPage($game) {
$gameDescriptionText = getContents('https://api.gog.com/products/' . $game->id . '?expand=description') or
die('Unable to get game description from GOG !');
$gameDescriptionText = getContents('https://api.gog.com/products/' . $game->id . '?expand=description')
or returnServerError('Unable to get game description from GOG !');
$gameDescriptionValue = json_decode($gameDescriptionText);

102
bridges/HaveIBeenPwnedBridge.php Normal file
View file

@ -0,0 +1,102 @@
<?php
class HaveIBeenPwnedBridge extends BridgeAbstract {
const NAME = 'Have I Been Pwned (HIBP) Bridge';
const URI = 'https://haveibeenpwned.com';
const DESCRIPTION = 'Returns list of Pwned websites';
const MAINTAINER = 'VerifiedJoseph';
const PARAMETERS = array(array(
'order' => array(
'name' => 'Order by',
'type' => 'list',
'values' => array(
'Breach date' => 'breachDate',
'Date added to HIBP' => 'dateAdded',
),
'defaultValue' => 'dateAdded',
)
));
const CACHE_TIMEOUT = 3600;
private $breachDateRegex = '/Breach date: ([0-9]{1,2} [A-Z-a-z]+ [0-9]{4})/';
private $dateAddedRegex = '/Date added to HIBP: ([0-9]{1,2} [A-Z-a-z]+ [0-9]{4})/';
private $accountsRegex = '/Compromised accounts: ([0-9,]+)/';
private $breaches = array();
public function collectData() {
$html = getSimpleHTMLDOM(self::URI . '/PwnedWebsites')
or returnServerError('Could not request: ' . self::URI . '/PwnedWebsites');
$breaches = array();
foreach($html->find('div.row') as $breach) {
$item = array();
if ($breach->class != 'row') {
continue;
}
preg_match($this->breachDateRegex, $breach->find('p', 1)->plaintext, $breachDate)
or returnServerError('Could not extract details');
preg_match($this->dateAddedRegex, $breach->find('p', 1)->plaintext, $dateAdded)
or returnServerError('Could not extract details');
preg_match($this->accountsRegex, $breach->find('p', 1)->plaintext, $accounts)
or returnServerError('Could not extract details');
$permalink = $breach->find('p', 1)->find('a', 0)->href;
// Remove permalink
$breach->find('p', 1)->find('a', 0)->outertext = '';
$item['title'] = $breach->find('h3', 0)->plaintext . ' - ' . $accounts[1] . ' breached accounts';
$item['dateAdded'] = strtotime($dateAdded[1]);
$item['breachDate'] = strtotime($breachDate[1]);
$item['uri'] = self::URI . '/PwnedWebsites' . $permalink;
$item['content'] = '<p>' . $breach->find('p', 0)->innertext . '<p>';
$item['content'] .= '<p>' . $breach->find('p', 1)->innertext . '<p>';
$this->breaches[] = $item;
}
$this->orderBreaches();
$this->createItems();
}
/**
* Order Breaches by date added or date breached
*/
private function orderBreaches() {
$sortBy = $this->getInput('order');
$sort = array();
foreach ($this->breaches as $key => $item) {
$sort[$key] = $item[$sortBy];
}
array_multisort($sort, SORT_DESC, $this->breaches);
}
/**
* Create items from breaches array
*/
private function createItems() {
foreach ($this->breaches as $breach) {
$item = array();
$item['title'] = $breach['title'];
$item['timestamp'] = $breach[$this->getInput('order')];
$item['uri'] = $breach['uri'];
$item['content'] = $breach['content'];
$this->items[] = $item;
}
}
}

60
bridges/MediapartBridge.php Normal file
View file

@ -0,0 +1,60 @@
<?php
class MediapartBridge extends FeedExpander {
const MAINTAINER = 'killruana';
const NAME = 'Mediapart Bridge';
const URI = 'https://www.mediapart.fr/';
const PARAMETERS = array(
array(
'single_page_mode' => array(
'name' => 'Single page article',
'type' => 'checkbox',
'title' => 'Display long articles on a single page',
'defaultValue' => 'checked'
),
'mpsessid' => array(
'name' => 'MPSESSID',
'type' => 'text',
'title' => 'Value of the session cookie MPSESSID'
)
)
);
const CACHE_TIMEOUT = 7200; // 2h
const DESCRIPTION = 'Returns the newest articles.';
public function collectData() {
$url = self::URI . 'articles/feed';
$this->collectExpandableDatas($url);
}
protected function parseItem($newsItem) {
$item = parent::parseItem($newsItem);
// Enable single page mode?
if ($this->getInput('single_page_mode') === true) {
$item['uri'] .= '?onglet=full';
}
// If a session cookie is defined, get the full article
$mpsessid = $this->getInput('mpsessid');
if (!empty($mpsessid)) {
// Set the session cookie
$opt = array();
$opt[CURLOPT_COOKIE] = 'MPSESSID=' . $mpsessid;
// Get the page
$articlePage = getSimpleHTMLDOM(
$newsItem->link . '?onglet=full',
array(),
$opt);
// Extract the article content
$content = $articlePage->find('div.content-article', 0)->innertext;
$content = sanitize($content);
$content = defaultLinkTo($content, static::URI);
$item['content'] .= $content;
}
return $item;
}
}

49
bridges/PikabuBridge.php
View file

@ -6,6 +6,16 @@ class PikabuBridge extends BridgeAbstract {
const DESCRIPTION = 'Выводит посты по тегу';
const MAINTAINER = 'em92';
const PARAMETERS_FILTER = array(
'name' => 'Фильтр',
'type' => 'list',
'values' => array(
'Горячее' => 'hot',
'Свежее' => 'new',
),
'defaultValue' => 'hot'
);
const PARAMETERS = array(
'По тегу' => array(
'tag' => array(
@ -13,21 +23,29 @@ class PikabuBridge extends BridgeAbstract {
'exampleValue' => 'it',
'required' => true
),
'filter' => array(
'name' => 'Фильтр',
'type' => 'list',
'values' => array(
'Горячее' => 'hot',
'Свежее' => 'new',
),
'defaultValue' => 'hot'
)
'filter' => self::PARAMETERS_FILTER
),
'По сообществу' => array(
'community' => array(
'name' => 'Сообщество',
'exampleValue' => 'linux',
'required' => true
),
'filter' => self::PARAMETERS_FILTER
)
);
protected $title = null;
public function getURI() {
if ($this->getInput('tag')) {
return self::URI . '/tag/' . rawurlencode($this->getInput('tag')) . '/' . rawurlencode($this->getInput('filter'));
} else if ($this->getInput('community')) {
$uri = self::URI . '/community/' . rawurlencode($this->getInput('community'));
if ($this->getInput('filter') != 'hot') {
$uri .= '/' . rawurlencode($this->getInput('filter'));
}
return $uri;
} else {
return parent::getURI();
}
@ -38,10 +56,10 @@ class PikabuBridge extends BridgeAbstract {
}
public function getName() {
if (is_string($this->getInput('tag'))) {
return $this->getInput('tag') . ' - ' . parent::getName();
} else {
if (is_null($this->title)) {
return parent::getName();
} else {
return $this->title . ' - ' . parent::getName();
}
}
@ -52,6 +70,8 @@ class PikabuBridge extends BridgeAbstract {
$text_html = iconv('windows-1251', 'utf-8', $text_html);
$html = str_get_html($text_html);
$this->title = $html->find('title', 0)->innertext;
foreach($html->find('article.story') as $post) {
$time = $post->find('time.story__datetime', 0);
if (is_null($time)) continue;
@ -67,6 +87,11 @@ class PikabuBridge extends BridgeAbstract {
}
}
foreach($post->find('[data-type=gifx]') as $el) {
$src = $el->getAttribute('data-source');
$el->outertext = '<img src="' . $src . '">';
}
foreach($post->find('img') as $img) {
$src = $img->getAttribute('src');
if (!$src) {

132
bridges/QPlayBridge.php Normal file
View file

@ -0,0 +1,132 @@
<?php
class QPlayBridge extends BridgeAbstract {
const NAME = 'Q Play';
const URI = 'https://www.qplay.pt';
const DESCRIPTION = 'Entretenimento e humor em Português';
const MAINTAINER = 'somini';
const PARAMETERS = array(
'Program' => array(
'program' => array(
'name' => 'Program Name',
'type' => 'text',
'required' => true,
),
),
'Catalog' => array(
'all_pages' => array(
'name' => 'All Pages',
'type' => 'checkbox',
'defaultValue' => false,
),
),
);
public function getIcon() {
# This should be the favicon served on `self::URI`
return 'https://s3.amazonaws.com/unode1/assets/4957/r3T9Lm9LTLmpAEX6FlSA_apple-touch-icon.png';
}
public function getURI() {
switch ($this->queriedContext) {
case 'Program':
return self::URI . '/programs/' . $this->getInput('program');
case 'Catalog':
return self::URI . '/catalog';
}
return parent::getURI();
}
public function getName() {
switch ($this->queriedContext) {
case 'Program':
$html = getSimpleHTMLDOMCached($this->getURI())
or returnServerError('Could not load content');
return $html->find('h1.program--title', 0)->innertext;
case 'Catalog':
return self::NAME . ' | Programas';
}
return parent::getName();
}
/* This uses the uscreen platform, other sites can adapt this. https://www.uscreen.tv/ */
public function collectData() {
switch ($this->queriedContext) {
case 'Program':
$program = $this->getInput('program');
$html = getSimpleHTMLDOMCached($this->getURI())
or returnServerError('Could not load content');
foreach($html->find('.cce--thumbnails-video-chapter') as $element) {
$cid = $element->getAttribute('data-id');
$item['title'] = $element->find('.cce--chapter-title', 0)->innertext;
$item['content'] = $element->find('.cce--thumbnails-image-block', 0)
. $element->find('.cce--chapter-body', 0)->innertext;
$item['uri'] = $this->getURI() . '?cid=' . $cid;
/* TODO: Suport login credentials? */
/* # Get direct video URL */
/* $json_source = getContents(self::URI . '/chapters/' . $cid, array('Cookie: _uscreen2_session=???;')) */
/* or returnServerError('Could not request chapter JSON'); */
/* $json = json_decode($json_source); */
/* $item['enclosures'] = [$json->fallback]; */
$this->items[] = $item;
}
break;
case 'Catalog':
$json_raw = getContents($this->getCatalogURI(1))
or returnServerError('Could not load catalog content');
$json = json_decode($json_raw);
$total_pages = $json->total_pages;
foreach($this->parseCatalogPage($json) as $item) {
$this->items[] = $item;
}
if ($this->getInput('all_pages') === true) {
foreach(range(2, $total_pages) as $page) {
$json_raw = getContents($this->getCatalogURI($page))
or returnServerError('Could not load catalog content (all pages)');
$json = json_decode($json_raw);
foreach($this->parseCatalogPage($json) as $item) {
$this->items[] = $item;
}
}
}
break;
}
}
private function getCatalogURI($page) {
return self::URI . '/catalog.json?page=' . $page;
}
private function parseCatalogPage($json) {
$items = array();
foreach($json->records as $record) {
$item = array();
$item['title'] = $record->title;
$item['content'] = $record->description
. '<div>Duration: ' . $record->duration . '</div>';
$item['timestamp'] = strtotime($record->release_date);
$item['uri'] = self::URI . $record->url;
$item['enclosures'] = array(
$record->main_poster,
);
$items[] = $item;
}
return $items;
}
}

47
bridges/RadioMelodieBridge.php
View file

@ -12,11 +12,12 @@ class RadioMelodieBridge extends BridgeAbstract {
public function collectData(){
$html = getSimpleHTMLDOM(self::URI . '/actu/')
or returnServerError('Could not request Radio Melodie.');
$list = $html->find('div[class=actu_col1]', 0)->children();;
$list = $html->find('div[class=displayList]', 0)->children();
foreach($list as $element) {
if($element->tag == 'a') {
$articleURL = self::URI . $element->href;
$article = getSimpleHTMLDOM($articleURL);
$textDOM = $article->find('article', 0);
// Initialise arrays
$item = array();
@ -24,52 +25,50 @@ class RadioMelodieBridge extends BridgeAbstract {
$picture = array();
// Get the Main picture URL
$picture[] = $this->rewriteImage($article->find('img[id=picturearticle]', 0)->src);
$audioHTML = $article->find('div[class=sm2-playlist-wrapper]');
$picture[] = $this->rewriteImage($article->find('div[id=pictureTitleSupport]', 0)->find('img', 0)->src);
$audioHTML = $article->find('audio');
// Remove the audio placeholder under the Audio player with an <audio>
// element and add the audio element to the enclosure
// Add the audio element to the enclosure
foreach($audioHTML as $audioElement) {
$audioURL = $audioElement->find('a', 0)->href;
$audioURL = $audioElement->src;
$audio[] = $audioURL;
$audioElement->outertext = '<audio controls src="' . $audioURL . '"></audio>';
$article->save();
}
// Rewrite pictures URL
$imgs = $article->find('img[src^="https://www.radiomelodie.com/image.php]');
$imgs = $textDOM->find('img[src^="http://www.radiomelodie.com/image.php]');
foreach($imgs as $img) {
$img->src = $this->rewriteImage($img->src);
$article->save();
}
// Remove inline audio player HTML
$inlinePlayers = $article->find('div[class*=sm2-main-controls]');
foreach($inlinePlayers as $inlinePlayer) {
$inlinePlayer->outertext = '';
$article->save();
}
// Remove Google Ads
$ads = $article->find('div[style^=margin:25px 0; position:relative; height:auto;]');
$ads = $article->find('div[class=adInline]');
foreach($ads as $ad) {
$ad->outertext = '';
$article->save();
}
$author = $article->find('div[id=author]', 0)->find('span', 0)->plaintext;
// Remove Radio Melodie Logo
$logoHTML = $article->find('div[id=logoArticleRM]', 0);
$logoHTML->outertext = '';
$article->save();
$author = $article->find('p[class=AuthorName]', 0)->plaintext;
$item['enclosures'] = array_merge($picture, $audio);
$item['author'] = $author;
$item['uri'] = $articleURL;
$item['title'] = $article->find('meta[property=og:title]', 0)->content;
$date_category = $article->find('div[class*=date]', 0)->plaintext;
$header = $article->find('a[class=fancybox]', 0)->innertext;
$textDOM = $article->find('div[class=text_content]', 0);
$textDOM->find('div[id=author]', 0)->outertext = '';
$date = $article->find('p[class*=date]', 0)->plaintext;
// Header Image
$header = '<img src="' . $picture[0] . '"/>';
// Remove the Date and Author part
$textDOM->find('div[class=AuthorDate]', 0)->outertext = '';
$article->save();
$text = $textDOM->innertext;
$item['content'] = '<h1>' . $item['title'] . '</h1>' . $date_category . $header . $text;
$item['content'] = '<h1>' . $item['title'] . '</h1>' . $date . '<br/>' . $header . $text;
$this->items[] = $item;
}
}
@ -81,7 +80,7 @@ class RadioMelodieBridge extends BridgeAbstract {
private function rewriteImage($url)
{
$parts = explode('?', $url);
parse_str($parts[1], $params);
parse_str(html_entity_decode($parts[1]), $params);
return self::URI . '/' . $params['image'];
}

5
bridges/Rue89Bridge.php
View file

@ -9,7 +9,7 @@ class Rue89Bridge extends BridgeAbstract {
public function collectData() {
$jsonArticles = getContents('https://appdata.nouvelobs.com/rue89/feed.json')
or die('Unable to query Rue89 !');
or returnServerError('Unable to query Rue89 !');
$articles = json_decode($jsonArticles)->items;
foreach($articles as $article) {
$this->items[] = $this->getArticle($article);
@ -19,7 +19,8 @@ class Rue89Bridge extends BridgeAbstract {
private function getArticle($articleInfo) {
$articleJson = getContents($articleInfo->json_url) or die('Unable to get article !');
$articleJson = getContents($articleInfo->json_url)
or returnServerError('Unable to get article !');
$article = json_decode($articleJson);
$item = array();
$item['title'] = $article->title;

12
bridges/SoundcloudBridge.php
View file

@ -16,6 +16,8 @@ class SoundCloudBridge extends BridgeAbstract {
const CLIENT_ID = 'W0KEWWILAjDiRH89X0jpwzuq6rbSK08R';
private $feedIcon = null;
public function collectData(){
$res = json_decode(getContents(
@ -25,6 +27,8 @@ class SoundCloudBridge extends BridgeAbstract {
. self::CLIENT_ID
)) or returnServerError('No results for this query');
$this->feedIcon = $res->avatar_url;
$tracks = json_decode(getContents(
'https://api.soundcloud.com/users/'
. urlencode($res->id)
@ -56,6 +60,14 @@ class SoundCloudBridge extends BridgeAbstract {
}
public function getIcon(){
if ($this->feedIcon) {
return $this->feedIcon;
}
return parent::getIcon();
}
public function getName(){
if(!is_null($this->getInput('u'))) {
return self::NAME . ' - ' . $this->getInput('u');

91
bridges/SteamBridge.php
View file

@ -8,44 +8,12 @@ class SteamBridge extends BridgeAbstract {
const MAINTAINER = 'jacknumber';
const PARAMETERS = array(
'Wishlist' => array(
'username' => array(
'name' => 'Username',
'userid' => array(
'name' => 'Steamid64 (find it on steamid.io)',
'title' => 'User ID (17 digits). Find your user ID with steamid.io or steamidfinder.com',
'required' => true,
),
'currency' => array(
'name' => 'Currency',
'type' => 'list',
'values' => array(
// source: http://steam.steamlytics.xyz/currencies
'USD' => 'us',
'GBP' => 'gb',
'EUR' => 'fr',
'CHF' => 'ch',
'RUB' => 'ru',
'BRL' => 'br',
'JPY' => 'jp',
'SEK' => 'se',
'IDR' => 'id',
'MYR' => 'my',
'PHP' => 'ph',
'SGD' => 'sg',
'THB' => 'th',
'KRW' => 'kr',
'TRY' => 'tr',
'MXN' => 'mx',
'CAD' => 'ca',
'NZD' => 'nz',
'CNY' => 'cn',
'INR' => 'in',
'CLP' => 'cl',
'PEN' => 'pe',
'COP' => 'co',
'ZAR' => 'za',
'HKD' => 'hk',
'TWD' => 'tw',
'SRD' => 'sr',
'AED' => 'ae',
),
'exampleValue' => '76561198821231205',
'pattern' => '[0-9]{17}',
),
'only_discount' => array(
'name' => 'Only discount',
@ -56,27 +24,15 @@ class SteamBridge extends BridgeAbstract {
public function collectData(){
$username = $this->getInput('username');
$params = array(
'cc' => $this->getInput('currency')
);
$userid = $this->getInput('userid');
$url = self::URI . 'wishlist/id/' . $username . '?' . http_build_query($params);
$targetVariable = 'g_rgAppInfo';
$sourceUrl = self::URI . 'wishlist/profiles/' . $userid . '/wishlistdata?p=0';
$sort = array();
$html = '';
$html = getSimpleHTMLDOM($url)
or returnServerError("Could not request Steam Wishlist. Tried:\n - $url");
$json = getContents($sourceUrl)
or returnServerError('Could not get content from wishlistdata (' . $sourceUrl . ')');
$jsContent = $html->find('.responsive_page_template_content script', 0)->innertext;
if(preg_match('/var ' . $targetVariable . ' = (.*?);/s', $jsContent, $matches)) {
$appsData = json_decode($matches[1]);
} else {
returnServerError("Could not parse JS variable ($targetVariable) in page content.");
}
$appsData = json_decode($json);
foreach($appsData as $id => $element) {
@ -87,6 +43,8 @@ class SteamBridge extends BridgeAbstract {
if($element->subs) {
$appIsBuyable = 1;
$priceBlock = str_get_html($element->subs[0]->discount_block);
$appPrice = str_replace('--', '00', $priceBlock->find('.discount_final_price', 0)->plaintext);
if($element->subs[0]->discount_pct) {
@ -94,8 +52,6 @@ class SteamBridge extends BridgeAbstract {
$discountBlock = str_get_html($element->subs[0]->discount_block);
$appDiscountValue = $discountBlock->find('.discount_pct', 0)->plaintext;
$appOldPrice = $discountBlock->find('.discount_original_price', 0)->plaintext;
$appNewPrice = $discountBlock->find('.discount_final_price', 0)->plaintext;
$appPrice = $appNewPrice;
} else {
@ -103,7 +59,6 @@ class SteamBridge extends BridgeAbstract {
continue;
}
$appPrice = $element->subs[0]->price / 100;
}
} else {
@ -117,11 +72,14 @@ class SteamBridge extends BridgeAbstract {
}
}
$coverUrl = str_replace('_292x136', '', strtok($element->capsule, '?'));
$picturesPath = pathinfo($coverUrl)['dirname'] . '/';
$item = array();
$item['uri'] = "http://store.steampowered.com/app/$id/";
$item['title'] = $element->name;
$item['type'] = $appType;
$item['cover'] = str_replace('_292x136', '', $element->capsule);
$item['cover'] = $coverUrl;
$item['timestamp'] = $element->added;
$item['isBuyable'] = $appIsBuyable;
$item['hasDiscount'] = $appHasDiscount;
@ -129,22 +87,29 @@ class SteamBridge extends BridgeAbstract {
$item['priority'] = $element->priority;
if($appIsBuyable) {
$item['price'] = floatval(str_replace(',', '.', $appPrice));
$item['content'] = $appPrice;
}
if($appIsFree) {
$item['content'] = 'Free';
}
if($appHasDiscount) {
$item['discount']['value'] = $appDiscountValue;
$item['discount']['oldPrice'] = floatval(str_replace(',', '.', $appOldPrice));
$item['discount']['newPrice'] = floatval(str_replace(',', '.', $appNewPrice));
$item['discount']['oldPrice'] = $appOldPrice;
$item['content'] = '<s>' . $appOldPrice . '</s> <b>' . $appPrice . '</b> (' . $appDiscountValue . ')';
}
$item['enclosures'] = array();
$item['enclosures'][] = str_replace('_292x136', '', $element->capsule);
$item['enclosures'][] = $coverUrl;
foreach($element->screenshots as $screenshot) {
$item['enclosures'][] = substr($element->capsule, 0, -31) . $screenshot;
foreach($element->screenshots as $screenshotFileName) {
$item['enclosures'][] = $picturesPath . $screenshotFileName;
}
$sort[$id] = $element->priority;

127
bridges/SteamCommunityBridge.php Normal file
View file

@ -0,0 +1,127 @@
<?php
class SteamCommunityBridge extends BridgeAbstract {
const NAME = 'Steam Community';
const URI = 'https://www.steamcommunity.com';
const DESCRIPTION = 'Get the latest community updates for a game on Steam.';
const MAINTAINER = 'thefranke';
const CACHE_TIMEOUT = 3600; // 1h
const PARAMETERS = array(
array(
'i' => array(
'name' => 'App ID',
'required' => true
),
'category' => array(
'name' => 'category',
'type' => 'list',
'exampleValue' => 'Artwork',
'title' => 'Select a category',
'values' => array(
'Artwork' => 'images',
'Screenshots' => 'screenshots',
'Videos' => 'videos'
)
)
)
);
public function getIcon() {
return self::URI . '/favicon.ico';
}
protected function getMainPage() {
$category = $this->getInput('category');
$html = getSimpleHTMLDOM($this->getURI() . '/?p=1&browsefilter=mostrecent')
or returnServerError('Could not fetch Steam data.');
return $html;
}
public function getName() {
$category = $this->getInput('category');
if (is_null('i') || is_null($category)) {
return self::NAME;
}
$html = $this->getMainPage();
$titleItem = $html->find('div.apphub_AppName', 0);
if (!$titleItem)
return self::NAME;
return $titleItem->innertext . ' (' . ucwords($category) . ')';
}
public function getURI() {
return self::URI . '/app/'
. $this->getInput('i') . '/'
. $this->getInput('category');
}
public function collectData() {
$category = $this->getInput('category');
$html = $this->getMainPage();
$cards = $html->find('div.apphub_Card');
foreach($cards as $card) {
$uri = $card->getAttribute('data-modal-content-url');
$htmlCard = getSimpleHTMLDOMCached($uri);
$author = $card->find('div.apphub_CardContentAuthorName', 0)->innertext;
$author = strip_tags($author);
$title = $author . '\'s screenshot';
if ($category != 'screenshots')
$title = $htmlCard->find('div.workshopItemTitle', 0)->innertext;
$date = $htmlCard->find('div.detailsStatRight', 0)->innertext;
// create item
$item = array();
$item['title'] = $title;
$item['uri'] = $uri;
$item['timestamp'] = strtotime($date);
$item['author'] = $author;
$item['categories'] = $category;
$media = $htmlCard->getElementById('ActualMedia');
$mediaURI = $media->getAttribute('src');
$downloadURI = $mediaURI;
if ($category == 'videos') {
preg_match('/.*\/embed\/(.*)\?/', $mediaURI, $result);
$youtubeID = $result[1];
$mediaURI = 'https://img.youtube.com/vi/' . $youtubeID . '/hqdefault.jpg';
$downloadURI = 'https://www.youtube.com/watch?v=' . $youtubeID;
}
$desc = '';
if ($category == 'screenshots') {
$descItem = $htmlCard->find('div.screenshotDescription', 0);
if ($descItem)
$desc = $descItem->innertext;
}
if ($category == 'images') {
$descItem = $htmlCard->find('div.nonScreenshotDescription', 0);
if ($descItem)
$desc = $descItem->innertext;
$downloadURI = $htmlCard->find('a.downloadImage', 0)->href;
}
$item['content'] = '<p><a href="' . $downloadURI . '"><img src="' . $mediaURI . '"/></a></p>';
$item['content'] .= '<p>' . $desc . '</p>';
$this->items[] = $item;
if (count($this->items) >= 10)
break;
}
}
}

2
bridges/VkBridge.php
View file

@ -52,7 +52,7 @@ class VkBridge extends BridgeAbstract
$text_html = $this->getContents()
or returnServerError('No results for group or user name "' . $this->getInput('u') . '".');
$text_html = iconv('windows-1251', 'utf-8', $text_html);
$text_html = iconv('windows-1251', 'utf-8//ignore', $text_html);
// makes album link generating work correctly
$text_html = str_replace('"class="page_album_link">', '" class="page_album_link">', $text_html);
$html = str_get_html($text_html);

10
caches/MemcachedCache.php
View file

@ -16,19 +16,19 @@ class MemcachedCache implements CacheInterface {
$host = Configuration::getConfig(get_called_class(), 'host');
$port = Configuration::getConfig(get_called_class(), 'port');
if (empty($host) && empty($port)) {
returnServerError('Configuration for ' . get_called_class() . ' missing. Please check your config.ini.php');
returnServerError('Configuration for ' . get_called_class() . ' missing. Please check your ' . FILE_CONFIG);
} else if (empty($host)) {
returnServerError('"host" param is not set for ' . get_called_class() . '. Please check your config.ini.php');
returnServerError('"host" param is not set for ' . get_called_class() . '. Please check your ' . FILE_CONFIG);
} else if (empty($port)) {
returnServerError('"port" param is not set for ' . get_called_class() . '. Please check your config.ini.php');
returnServerError('"port" param is not set for ' . get_called_class() . '. Please check your ' . FILE_CONFIG);
} else if (!ctype_digit($port)) {
returnServerError('"port" param is invalid for ' . get_called_class() . '. Please check your config.ini.php');
returnServerError('"port" param is invalid for ' . get_called_class() . '. Please check your ' . FILE_CONFIG);
}
$port = intval($port);
if ($port < 1 || $port > 65535) {
returnServerError('"port" param is invalid for ' . get_called_class() . '. Please check your config.ini.php');
returnServerError('"port" param is invalid for ' . get_called_class() . '. Please check your ' . FILE_CONFIG);
}
$conn = new Memcached();

4
caches/SQLiteCache.php
View file

@ -15,12 +15,12 @@ class SQLiteCache implements CacheInterface {
$file = Configuration::getConfig(get_called_class(), 'file');
if (empty($file)) {
die('Configuration for ' . get_called_class() . ' missing. Please check your config.ini.php');
die('Configuration for ' . get_called_class() . ' missing. Please check your ' . FILE_CONFIG);
}
if (dirname($file) == '.') {
$file = PATH_CACHE . $file;
} elseif (!is_dir(dirname($file))) {
die('Invalid configuration for ' . get_called_class() . '. Please check your config.ini.php');
die('Invalid configuration for ' . get_called_class() . '. Please check your ' . FILE_CONFIG);
}
if (!is_file($file)) {

12
composer.json Normal file
View file

@ -0,0 +1,12 @@
{
"require": {
"php": ">=5.6",
"ext-mbstring": "*",
"ext-sqlite3": "*",
"ext-curl": "*",
"ext-openssl": "*",
"ext-libxml": "*",
"ext-simplexml": "*",
"ext-json": "*"
}
}

26
composer.lock generated Normal file
View file

@ -0,0 +1,26 @@
{
"_readme": [
"This file locks the dependencies of your project to a known state",
"Read more about it at https://getcomposer.org/doc/01-basic-usage.md#installing-dependencies",
"This file is @generated automatically"
],
"content-hash": "ef341ee18f28c7bd5832e188fe157734",
"packages": [],
"packages-dev": [],
"aliases": [],
"minimum-stability": "stable",
"stability-flags": [],
"prefer-stable": false,
"prefer-lowest": false,
"platform": {
"php": ">=5.6",
"ext-mbstring": "*",
"ext-sqlite3": "*",
"ext-curl": "*",
"ext-openssl": "*",
"ext-libxml": "*",
"ext-simplexml": "*",
"ext-json": "*"
},
"platform-dev": []
}

8
config.default.ini.php
View file

@ -4,6 +4,14 @@
; file, it will be replaced on the next update of RSS-Bridge! You can specify
; your own configuration in 'config.ini.php' (copy this file).
[system]
; Defines the timezone used by RSS-Bridge
; Find a list of supported timezones at
; https://www.php.net/manual/en/timezones.php
; timezone = "UTC" (default)
timezone = "UTC"
[cache]
; Defines the cache type used by RSS-Bridge

21
index.php
View file

@ -6,8 +6,6 @@ Configuration::loadConfiguration();
Authentication::showPromptIfNeeded();
date_default_timezone_set('UTC');
/*
Move the CLI arguments to the $_GET array, in order to be able to use
rss-bridge from the command line
@ -29,27 +27,8 @@ define('USER_AGENT',
ini_set('user_agent', USER_AGENT);
// default whitelist
$whitelist_default = array(
'BandcampBridge',
'CryptomeBridge',
'DansTonChatBridge',
'DuckDuckGoBridge',
'FacebookBridge',
'FlickrBridge',
'GoogleSearchBridge',
'IdenticaBridge',
'InstagramBridge',
'OpenClassroomsBridge',
'PinterestBridge',
'ScmbBridge',
'TwitterBridge',
'WikipediaBridge',
'YoutubeBridge');
try {
Bridge::setWhitelist($whitelist_default);
$actionFac = new \ActionFactory();
$actionFac->setWorkingDir(PATH_LIB_ACTIONS);

31
lib/Bridge.php
View file

@ -192,7 +192,8 @@ class Bridge {
/**
* Returns the whitelist.
*
* On first call this function reads the whitelist from {@see WHITELIST}.
* On first call this function reads the whitelist from {@see WHITELIST} if
* the file exists, {@see WHITELIST_DEFAULT} otherwise.
* * Each line in the file specifies one bridge on the whitelist.
* * An empty file disables all bridges.
* * If the file only only contains `*`, all bridges are whitelisted.
@ -210,19 +211,21 @@ class Bridge {
if($firstCall) {
// Create initial whitelist or load from disk
if (!file_exists(WHITELIST) && !empty(self::$whitelist)) {
file_put_contents(WHITELIST, implode("\n", self::$whitelist));
} elseif(file_exists(WHITELIST)) {
if(file_exists(WHITELIST)) {
$contents = trim(file_get_contents(WHITELIST));
} elseif(file_exists(WHITELIST_DEFAULT)) {
$contents = trim(file_get_contents(WHITELIST_DEFAULT));
} else {
$contents = '';
}
if($contents === '*') { // Whitelist all bridges
self::$whitelist = self::getBridgeNames();
} else {
self::$whitelist = array_map('self::sanitizeBridgeName', explode("\n", $contents));
if($contents === '*') { // Whitelist all bridges
self::$whitelist = self::getBridgeNames();
} else {
//self::$whitelist = array_map('self::sanitizeBridgeName', explode("\n", $contents));
foreach(explode("\n", $contents) as $bridgeName) {
self::$whitelist[] = self::sanitizeBridgeName($bridgeName);
}
}
}
@ -280,6 +283,12 @@ class Bridge {
$name = $matches[1];
}
// Improve performance for correctly written bridge names
if(in_array($name, self::getBridgeNames())) {
$index = array_search($name, self::getBridgeNames());
return self::getBridgeNames()[$index];
}
// The name is valid if a corresponding bridge file is found on disk
if(in_array(strtolower($name), array_map('strtolower', self::getBridgeNames()))) {
$index = array_search(strtolower($name), array_map('strtolower', self::getBridgeNames()));

114
lib/Configuration.php
View file

@ -28,7 +28,7 @@ final class Configuration {
*
* @todo Replace this property by a constant.
*/
public static $VERSION = 'dev.2019-05-08';
public static $VERSION = '2019-06-08';
/**
* Holds the configuration data.
@ -80,35 +80,31 @@ final class Configuration {
// Check PHP version
if(version_compare(PHP_VERSION, '5.6.0') === -1)
die('RSS-Bridge requires at least PHP version 5.6.0!');
self::reportError('RSS-Bridge requires at least PHP version 5.6.0!');
// extensions check
if(!extension_loaded('openssl'))
die('"openssl" extension not loaded. Please check "php.ini"');
self::reportError('"openssl" extension not loaded. Please check "php.ini"');
if(!extension_loaded('libxml'))
die('"libxml" extension not loaded. Please check "php.ini"');
self::reportError('"libxml" extension not loaded. Please check "php.ini"');
if(!extension_loaded('mbstring'))
die('"mbstring" extension not loaded. Please check "php.ini"');
self::reportError('"mbstring" extension not loaded. Please check "php.ini"');
if(!extension_loaded('simplexml'))
die('"simplexml" extension not loaded. Please check "php.ini"');
self::reportError('"simplexml" extension not loaded. Please check "php.ini"');
// Allow RSS-Bridge to run without curl module in CLI mode without root certificates
if(!extension_loaded('curl') && !(php_sapi_name() === 'cli' && empty(ini_get('curl.cainfo'))))
die('"curl" extension not loaded. Please check "php.ini"');
self::reportError('"curl" extension not loaded. Please check "php.ini"');
if(!extension_loaded('json'))
die('"json" extension not loaded. Please check "php.ini"');
self::reportError('"json" extension not loaded. Please check "php.ini"');
// Check cache folder permissions (write permissions required)
if(!is_writable(PATH_CACHE))
die('RSS-Bridge does not have write permissions for ' . PATH_CACHE . '!');
// Check whitelist file permissions
if(!file_exists(WHITELIST) && !is_writable(dirname(WHITELIST)))
die('RSS-Bridge does not have write permissions for ' . WHITELIST . '!');
self::reportError('RSS-Bridge does not have write permissions for ' . PATH_CACHE . '!');
}
@ -118,15 +114,13 @@ final class Configuration {
* Returns an error message and aborts execution if the configuration is invalid.
*
* The RSS-Bridge configuration is split into two files:
* - `config.default.ini.php`: The default configuration file that ships with
* every release of RSS-Bridge (do not modify this file!).
* - `config.ini.php`: The local configuration file that can be modified by
* server administrators.
* - {@see FILE_CONFIG_DEFAULT} The default configuration file that ships
* with every release of RSS-Bridge (do not modify this file!).
* - {@see FILE_CONFIG} The local configuration file that can be modified
* by server administrators.
*
* The files must be located at {@see PATH_ROOT}
*
* RSS-Bridge will first load `config.default.ini.php` into memory and then
* replace parameters with the contents of `config.ini.php`. That way new
* RSS-Bridge will first load {@see FILE_CONFIG_DEFAULT} into memory and then
* replace parameters with the contents of {@see FILE_CONFIG}. That way new
* parameters are automatically initialized with default values and custom
* configurations can be reduced to the minimum set of parametes necessary
* (only the ones that changed).
@ -140,16 +134,16 @@ final class Configuration {
*/
public static function loadConfiguration() {
if(!file_exists(PATH_ROOT . 'config.default.ini.php'))
die('The default configuration file "config.default.ini.php" is missing!');
if(!file_exists(FILE_CONFIG_DEFAULT))
self::reportError('The default configuration file is missing at ' . FILE_CONFIG_DEFAULT);
Configuration::$config = parse_ini_file(PATH_ROOT . 'config.default.ini.php', true, INI_SCANNER_TYPED);
Configuration::$config = parse_ini_file(FILE_CONFIG_DEFAULT, true, INI_SCANNER_TYPED);
if(!Configuration::$config)
die('Error parsing config.default.ini.php');
self::reportError('Error parsing ' . FILE_CONFIG_DEFAULT);
if(file_exists(PATH_ROOT . 'config.ini.php')) {
if(file_exists(FILE_CONFIG)) {
// Replace default configuration with custom settings
foreach(parse_ini_file(PATH_ROOT . 'config.ini.php', true, INI_SCANNER_TYPED) as $header => $section) {
foreach(parse_ini_file(FILE_CONFIG, true, INI_SCANNER_TYPED) as $header => $section) {
foreach($section as $key => $value) {
// Skip unknown sections and keys
if(array_key_exists($header, Configuration::$config) && array_key_exists($key, Configuration::$config[$header])) {
@ -159,8 +153,14 @@ final class Configuration {
}
}
if(!is_string(self::getConfig('system', 'timezone'))
|| !in_array(self::getConfig('system', 'timezone'), timezone_identifiers_list(DateTimeZone::ALL_WITH_BC)))
self::reportConfigurationError('system', 'timezone');
date_default_timezone_set(self::getConfig('system', 'timezone'));
if(!is_string(self::getConfig('proxy', 'url')))
die('Parameter [proxy] => "url" is not a valid string! Please check "config.ini.php"!');
self::reportConfigurationError('proxy', 'url', 'Is not a valid string');
if(!empty(self::getConfig('proxy', 'url'))) {
/** URL of the proxy server */
@ -168,38 +168,38 @@ final class Configuration {
}
if(!is_bool(self::getConfig('proxy', 'by_bridge')))
die('Parameter [proxy] => "by_bridge" is not a valid Boolean! Please check "config.ini.php"!');
self::reportConfigurationError('proxy', 'by_bridge', 'Is not a valid Boolean');
/** True if proxy usage can be enabled selectively for each bridge */
define('PROXY_BYBRIDGE', self::getConfig('proxy', 'by_bridge'));
if(!is_string(self::getConfig('proxy', 'name')))
die('Parameter [proxy] => "name" is not a valid string! Please check "config.ini.php"!');
self::reportConfigurationError('proxy', 'name', 'Is not a valid string');
/** Name of the proxy server */
define('PROXY_NAME', self::getConfig('proxy', 'name'));
if(!is_string(self::getConfig('cache', 'type')))
die('Parameter [cache] => "type" is not a valid string! Please check "config.ini.php"!');
self::reportConfigurationError('cache', 'type', 'Is not a valid string');
if(!is_bool(self::getConfig('cache', 'custom_timeout')))
die('Parameter [cache] => "custom_timeout" is not a valid Boolean! Please check "config.ini.php"!');
self::reportConfigurationError('cache', 'custom_timeout', 'Is not a valid Boolean');
/** True if the cache timeout can be specified by the user */
define('CUSTOM_CACHE_TIMEOUT', self::getConfig('cache', 'custom_timeout'));
if(!is_bool(self::getConfig('authentication', 'enable')))
die('Parameter [authentication] => "enable" is not a valid Boolean! Please check "config.ini.php"!');
self::reportConfigurationError('authentication', 'enable', 'Is not a valid Boolean');
if(!is_string(self::getConfig('authentication', 'username')))
die('Parameter [authentication] => "username" is not a valid string! Please check "config.ini.php"!');
self::reportConfigurationError('authentication', 'username', 'Is not a valid string');
if(!is_string(self::getConfig('authentication', 'password')))
die('Parameter [authentication] => "password" is not a valid string! Please check "config.ini.php"!');
self::reportConfigurationError('authentication', 'password', 'Is not a valid string');
if(!empty(self::getConfig('admin', 'email'))
&& !filter_var(self::getConfig('admin', 'email'), FILTER_VALIDATE_EMAIL))
die('Parameter [admin] => "email" is not a valid email address! Please check "config.ini.php"!');
self::reportConfigurationError('admin', 'email', 'Is not a valid email address');
}
@ -246,4 +246,46 @@ final class Configuration {
return Configuration::$VERSION;
}
/**
* Reports an configuration error for the specified section and key to the
* user and ends execution
*
* @param string $section The section name
* @param string $key The configuration key
* @param string $message An optional message to the user
*
* @return void
*/
private static function reportConfigurationError($section, $key, $message = '') {
$report = "Parameter [{$section}] => \"{$key}\" is invalid!" . PHP_EOL;
if(file_exists(FILE_CONFIG)) {
$report .= 'Please check your configuration file at ' . FILE_CONFIG . PHP_EOL;
} elseif(!file_exists(FILE_CONFIG_DEFAULT)) {
$report .= 'The default configuration file is missing at ' . FILE_CONFIG_DEFAULT . PHP_EOL;
} else {
$report .= 'The default configuration file is broken.' . PHP_EOL
. 'Restore the original file from ' . REPOSITORY . PHP_EOL;
}
$report .= $message;
self::reportError($report);
}
/**
* Reports an error message to the user and ends execution
*
* @param string $message The error message
*
* @return void
*/
private static function reportError($message) {
header('Content-Type: text/plain', true, 500);
die('Configuration error' . PHP_EOL . $message);
}
}

26
lib/Exceptions.php
View file

@ -11,6 +11,15 @@
* @link https://github.com/rss-bridge/rss-bridge
*/
/**
* Builds a GitHub search query to find open bugs for the current bridge
*/
function buildGitHubSearchQuery($bridgeName){
return REPOSITORY
. 'issues?q='
. urlencode('is:issue is:open ' . $bridgeName);
}
/**
* Returns an URL that automatically populates a new issue on GitHub based
* on the information provided
@ -83,7 +92,8 @@ function buildBridgeException($e, $bridge){
. '`';
$body_html = nl2br($body);
$link = buildGitHubIssueQuery($title, $body, 'bug report', $bridge->getMaintainer());
$link = buildGitHubIssueQuery($title, $body, 'Bridge-Broken', $bridge->getMaintainer());
$searchQuery = buildGitHubSearchQuery($bridge::NAME);
$header = buildHeader($e, $bridge);
$message = <<<EOD
@ -91,7 +101,7 @@ function buildBridgeException($e, $bridge){
remote website's content!<br>
{$body_html}
EOD;
$section = buildSection($e, $bridge, $message, $link);
$section = buildSection($e, $bridge, $message, $link, $searchQuery);
return $section;
}
@ -119,11 +129,12 @@ function buildTransformException($e, $bridge){
. (isset($_SERVER['QUERY_STRING']) ? $_SERVER['QUERY_STRING'] : '')
. '`';
$link = buildGitHubIssueQuery($title, $body, 'bug report', $bridge->getMaintainer());
$link = buildGitHubIssueQuery($title, $body, 'Bridge-Broken', $bridge->getMaintainer());
$searchQuery = buildGitHubSearchQuery($bridge::NAME);
$header = buildHeader($e, $bridge);
$message = "RSS-Bridge was unable to transform the contents returned by
<strong>{$bridge->getName()}</strong>!";
$section = buildSection($e, $bridge, $message, $link);
$section = buildSection($e, $bridge, $message, $link, $searchQuery);
return buildPage($title, $header, $section);
}
@ -154,11 +165,12 @@ EOD;
* @param object $bridge The bridge object
* @param string $message The message to display
* @param string $link The link to include in the anchor
* @param string $searchQuery A GitHub search query for the current bridge
* @return string The HTML section
*
* @todo This function belongs inside a class
*/
function buildSection($e, $bridge, $message, $link){
function buildSection($e, $bridge, $message, $link, $searchQuery){
return <<<EOD
<section>
<p class="exception-message">{$message}</p>
@ -166,9 +178,13 @@ function buildSection($e, $bridge, $message, $link){
<ul class="advice">
<li>Press Return to check your input parameters</li>
<li>Press F5 to retry</li>
<li>Check if this issue was already reported on <a href="{$searchQuery}">GitHub</a> (give it a thumbs-up)</li>
<li>Open a <a href="{$link}">GitHub Issue</a> if this error persists</li>
</ul>
</div>
<a href="{$searchQuery}" title="Opens GitHub to search for similar issues">
<button>Search GitHub Issues</button>
</a>
<a href="{$link}" title="After clicking this button you can review
the issue before submitting it"><button>Open GitHub Issue</button></a>
<p class="maintainer">{$bridge->getMaintainer()}</p>

3
lib/FeedItem.php
View file

@ -418,6 +418,9 @@ class FeedItem {
if(!is_string($uid)) {
Debug::log('Unique id must be a string!');
} elseif (preg_match('/^[a-f0-9]{40}$/', $uid)) {
// keep id if it already is a SHA-1 hash
$this->uid = $uid;
} else {
$this->uid = sha1($uid);
}

26
lib/html.php
View file

@ -32,18 +32,7 @@ function sanitize($html,
$htmlContent = str_get_html($html);
/*
* Notice: simple_html_dom currently doesn't support "->find(*)", which is a
* known issue: https://sourceforge.net/p/simplehtmldom/bugs/157/
*
* A solution to this is to find all nodes WITHOUT a specific attribute. If
* the attribute is very unlikely to appear in the DOM, this is essentially
* returning all nodes.
*
* "*[!b38fd2b1fe7f4747d6b1c1254ccd055e]" is doing exactly that. The attrib
* "b38fd2b1fe7f4747d6b1c1254ccd055e" is very unlikely to appear in any DOM.
*/
foreach($htmlContent->find('*[!b38fd2b1fe7f4747d6b1c1254ccd055e]') as $element) {
foreach($htmlContent->find('*') as $element) {
if(in_array($element->tag, $text_to_keep)) {
$element->outertext = $element->plaintext;
} elseif(in_array($element->tag, $tags_to_remove)) {
@ -90,18 +79,7 @@ function backgroundToImg($htmlContent) {
$regex = '/background-image[ ]{0,}:[ ]{0,}url\([\'"]{0,}(.*?)[\'"]{0,}\)/';
$htmlContent = str_get_html($htmlContent);
/*
* Notice: simple_html_dom currently doesn't support "->find(*)", which is a
* known issue: https://sourceforge.net/p/simplehtmldom/bugs/157/
*
* A solution to this is to find all nodes WITHOUT a specific attribute. If
* the attribute is very unlikely to appear in the DOM, this is essentially
* returning all nodes.
*
* "*[!b38fd2b1fe7f4747d6b1c1254ccd055e]" is doing exactly that. The attrib
* "b38fd2b1fe7f4747d6b1c1254ccd055e" is very unlikely to appear in any DOM.
*/
foreach($htmlContent->find('*[!b38fd2b1fe7f4747d6b1c1254ccd055e]') as $element) {
foreach($htmlContent->find('*') as $element) {
if(preg_match($regex, $element->style, $matches) > 0) {

25
lib/rssbridge.php
View file

@ -15,28 +15,37 @@
define('PATH_ROOT', __DIR__ . '/../');
/** Path to the core library */
define('PATH_LIB', __DIR__ . '/../lib/'); // Path to core library
define('PATH_LIB', PATH_ROOT . 'lib/');
/** Path to the vendor library */
define('PATH_LIB_VENDOR', __DIR__ . '/../vendor/');
define('PATH_LIB_VENDOR', PATH_ROOT . 'vendor/');
/** Path to the bridges library */
define('PATH_LIB_BRIDGES', __DIR__ . '/../bridges/');
define('PATH_LIB_BRIDGES', PATH_ROOT . 'bridges/');
/** Path to the formats library */
define('PATH_LIB_FORMATS', __DIR__ . '/../formats/');
define('PATH_LIB_FORMATS', PATH_ROOT . 'formats/');
/** Path to the caches library */
define('PATH_LIB_CACHES', __DIR__ . '/../caches/');
define('PATH_LIB_CACHES', PATH_ROOT . 'caches/');
/** Path to the actions library */
define('PATH_LIB_ACTIONS', __DIR__ . '/../actions/');
define('PATH_LIB_ACTIONS', PATH_ROOT . 'actions/');
/** Path to the cache folder */
define('PATH_CACHE', __DIR__ . '/../cache/');
define('PATH_CACHE', PATH_ROOT . 'cache/');
/** Path to the whitelist file */
define('WHITELIST', __DIR__ . '/../whitelist.txt');
define('WHITELIST', PATH_ROOT . 'whitelist.txt');
/** Path to the default whitelist file */
define('WHITELIST_DEFAULT', PATH_ROOT . 'whitelist.default.txt');
/** Path to the configuration file */
define('FILE_CONFIG', PATH_ROOT . 'config.ini.php');
/** Path to the default configuration file */
define('FILE_CONFIG_DEFAULT', PATH_ROOT . 'config.default.ini.php');
/** URL to the RSS-Bridge repository */
define('REPOSITORY', 'https://github.com/RSS-Bridge/rss-bridge/');

13
static/style.css
View file

@ -84,6 +84,12 @@ input[type="number"]:focus {
border-color: #888;
}
input:focus::-webkit-input-placeholder { opacity: 0; }
input:focus::-moz-placeholder { opacity: 0; }
input:focus::placeholder { opacity: 0; }
input:focus:-moz-placeholder { opacity: 0; }
input:focus:-ms-input-placeholder { opacity: 0; }
.searchbar {
width: 40%;
margin: 40px auto 100px;
@ -101,13 +107,6 @@ input[type="number"]:focus {
text-align: center;
}
.searchbar input[type="text"]:focus::-webkit-input-placeholder,
.searchbar input[type="text"]:focus::-moz-placeholder,
.searchbar input[type="text"]:focus:-moz-placeholder,
.searchbar input[type="text"]:focus:-ms-input-placeholder {
opacity: 0;
}
.searchbar > h3 {
font-size: 200%;
font-weight: bold;

21
vendor/simplehtmldom/LICENSE vendored Normal file
View file

@ -0,0 +1,21 @@
MIT License
Copyright (c) 2019 S.C. Chen, John Schlick, logmanoriginal
Permission is hereby granted, free of charge, to any person obtaining a copy
of this software and associated documentation files (the "Software"), to deal
in the Software without restriction, including without limitation the rights
to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
copies of the Software, and to permit persons to whom the Software is
furnished to do so, subject to the following conditions:
The above copyright notice and this permission notice shall be included in all
copies or substantial portions of the Software.
THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE
SOFTWARE.

1020
vendor/simplehtmldom/simple_html_dom.php vendored
View file

File diff suppressed because it is too large Load diff

15
whitelist.default.txt Normal file
View file

@ -0,0 +1,15 @@
Bandcamp
Cryptome
DansTonChat
DuckDuckGo
Facebook
Flickr
GoogleSearch
Identica
Instagram
OpenClassrooms
Pinterest
Scmb
Twitter
Wikipedia
Youtube