Compare commits

...

192 Commits

Author SHA1 Message Date
Knah Tsaeb c65568437f Merge branch 'master' into kt_bridge 2019-12-10 12:34:36 +01:00
Joe Digilio ad661c4c91 [RedditBridge] Fix typo prevents bridge from working (#1383) 2019-12-05 18:07:50 +01:00
Grégory T ba8c4623ed [DisplayAction] Fix function call on a member (add ->) (#1379) 2019-12-04 18:34:26 +01:00
logmanoriginal ba43c87952 [RevolutBridge] Remove bridge
An official RSS feed is available at https://blog.revolut.com/rss/

Note that there is also an invisible "RSS" button next to the Facebook
and Twitter icons at the menu bar.

References #1321
2019-12-04 18:23:13 +01:00
Grégory T 595b87946d [TorrentGalaxyBridge] Add new bridge (#1378) 2019-12-02 20:31:50 +01:00
logmanoriginal 99d4e1a43d Bump version to dev.2019-12-01 2019-12-01 13:40:17 +01:00
logmanoriginal 477de4e2df Bump version to 2019-12-01 2019-12-01 13:34:09 +01:00
logmanoriginal 246470da18 [README] Update list of contributors 2019-12-01 13:33:03 +01:00
LogMANOriginal df9f7eb778
[FacebookBridge] Fix permalink issue (#1358)
Facebook has changed their strategy regarding permalinks, which
now include lots of unnecessary target data. Fortunately it also
contains the unique story id which we can utilize as URI.
2019-12-01 13:24:11 +01:00
David 375831f516 [NineGagBridge] Add filter option for animated content (#1374) 2019-12-01 13:07:25 +01:00
Roliga e518936be7 [SoundcloudBridge] Automatically acquire client_id (#1375)
Also some slight refactoring, as well as adding Roliga as maintainer.
2019-12-01 12:42:53 +01:00
Jacob Mansfield 583dfb4958 [TheWhiteboard] Create new bridge for The Whiteboard (#1327) 2019-12-01 11:30:16 +01:00
Jacob Mansfield 2de45b163e [FurAffinityUser] Add new bridge (#1326) 2019-12-01 11:27:41 +01:00
Shuto Yano 48b0164676 [InstagramBridge] Fix instagram GraphSidecar output and Video embedding (#1361)
* [InstagramBridge] Fix GraphSidecar output

Fix following issues which related to output of the GraphSidecar type posts.
- The GraphSidecar post's media wasn't outputted except for first picture when searching by hashtag or location
- Video didn't embedded
NOTE:
The function getInstagramStory() which was called when the post type is GraphSidecar didn't seem to work just as one intended.
Because the web request called in that function is just to get the media of single post, NOT to get the media of Story.
But I don't have any idea to solve #694, so it seems be better to rename these function and member variable properly.
2019-12-01 11:25:20 +01:00
somini 4b3c3c58d2 [DiárioDoAlentejo] Add new bridge (#1360) 2019-12-01 11:18:45 +01:00
somini 60768b4885 [DisplayAction] Don't return redirect error codes (#1359)
This might lead to redirect loops.

See
https://github.com/RSS-Bridge/rss-bridge/pull/1071#issuecomment-515632848

Cherry-picked from eb21d6f.
2019-12-01 11:13:57 +01:00
Eugene Molotov 02dd778124 [VkBridge] Save internal links in posts and get hashtags before making item (#1363) 2019-11-18 10:51:35 +01:00
Eugene Molotov 5b63121e92 [VkBridge] Change access token (#1357)
Previous access token was revoked
2019-11-09 18:51:16 +01:00
St. John Johnson 49019a843f [ComicsKingdomBridge] Add new bridge (#1353) 2019-11-09 18:50:08 +01:00
Roliga d65714fa47 [HtmlFormat] Add syndication links (#1348)
Adds <link> elements for each additional output format in the <head> of
HTML format output to allow RSS readers to find the actual feeds
directly from the HTML page.
2019-11-09 18:43:21 +01:00
Joseph 8161829ad5 [OpenwhydBridge] Add new bridge (#1338)
Rename WhydBridge to OpenwhydBridge
2019-11-09 18:36:50 +01:00
Nemo 7f35fc9f6b [AppleAppStoreBridge] Add new bridge (#1316)
This bridge allows you to follow iOS/iPad application updates
directly over RSS. This allows you to get notified of an application
update even if you don't have a iDevice. This may be useful in cases
where you want to be notified of a competitor's application for eg.

I built this after I got tired of waiting for WhatsApp to push their
security release on iOS. It shows up on the AppStore 7 days later
2019-11-09 18:28:00 +01:00
logmanoriginal 3bc8c9468a phpcs: Always use long array syntax
Most of the code in RSS-Bridge uses the long array syntax.
This commit adds a check to enforce using this syntax over
the short array syntax.

All failures have been fixed.
2019-11-01 18:06:55 +01:00
logmanoriginal 1df3598a74 [Dockerfile] Drop minimum security level back to TLS 1.0
Debian increased the minimum security level for OpenSSL from TLS 1.0
to TLS 1.2 [1] which also affects the Debian-based PHP image for Docker.

This change can break some bridges which have to connect to servers with
lower security level. Since all browsers still connect to these servers,
so should RSS-Bridge.

Note that according to [2] Mozilla, Firefox, Microsoft, Google and Apple
plan to increase the minimum security level to TLS 1.2 around March 2020.
At this time RSS-Bridge should follow the browser changes.

This commit updates the Dockerfile to automatically drop the minimum
security level back to TLS 1.0.

Based on the solution provided by @theScrabi in #1318

[1] https://wiki.debian.org/ContinuousIntegration/TriagingTips/openssl-1.1.1
[2] 553fc8e61f/debian/libssl1.1.NEWS
2019-11-01 17:12:45 +01:00
logmanoriginal 5f64fe2516 [BridgeAbstract] Fix broken assignment of defaultValue
setInputs() currently looks if the global array defines a 'value'
for a given parameter, but that isn't supported by the API. It
needs to be 'defaultValue'.
2019-11-01 15:29:16 +01:00
logmanoriginal 50eee7e7b3 [KununuBridge] Add feed item limit
This bridge currently takes a very long time to process
all news items on the page, when in many cases only one
or two had been added since the last check.

This commit adds a new parameter 'limit', which defines
the maximum number of items to add to the feed. This is
an optional paramter that defaults to 3.
2019-11-01 15:27:35 +01:00
logmanoriginal c0df9815c7 [DesoutterBridge] Add feed item limit
This bridge currently takes a very long time to process
all news items on the page, when in many cases only one
or two had been added since the last check.

This commit adds a new parameter 'limit', which defines
the maximum number of items to add to the feed. This is
an optional paramter that defaults to 3.
2019-11-01 15:07:25 +01:00
Léo Maradan 46d5895d1d [RedditBridge] Add new bridge (#1213) 2019-11-01 13:54:03 +01:00
Anchit Bajaj 7c16aaf303 [VarietyBridge] Add new bridge (#1307) 2019-11-01 13:48:09 +01:00
LogMANOriginal cdc1d9c9ba
action: Add action to check bridge connectivity (#1147)
* action: Add action to check bridge connectivity

It is currently not simply possible to check if the remote
server for a bridge is reachable or not, which means some
of the bridges might no longer work because the server is
no longer on the internet.

In order to find those bridges we can either check each
bridge individually (which takes a lot of effort), or use
an automated script to do this for us.

If a server is no longer reachable it could mean that it is
temporarily unavailable, or shutdown permanently. The results
of this script will at least help identifying such servers.

* [Connectivity] Use Bootstrap container to properly display contents

* [Connectivity] Limit connectivity checks to debug mode

Connectivity checks take a long time to execute and can require a lot
of bandwidth. Therefore, administrators should be able to determine
when and who is able to utilize this action. The best way to prevent
regular users from accessing this action is by making it available in
debug mode only (public servers should never run in debug mode anyway).

* [Connectivity] Split implemenation into multiple files

* [Connectivity] Make web page responsive to user input

* [Connectivity] Make status message sticky

* [Connectivity] Add icon to the status message

* [contents] Add the ability for getContents to return header information

* [Connectivity] Add header information to the reply Json data

* [Connectivity] Add new status (blue) for redirected sites

Also adds titles to status icons (Successful, Redirected, Inactive, Failed)

* [Connectivity] Fix show doesn't work for inactive bridges

* [Connectivity] Fix typo

* [Connectivity] Catch errors in promise chains

* [Connectivity] Allow search by status and update dynamically

* [Connectivity] Add a progress bar

* [Connectivity] Use bridge factory

* [Connectivity] Import Bootstrap v4.3.1 CSS
2019-10-31 22:02:38 +01:00
LogMANOriginal 6bc83310b9
core: Add info button for input fields with title (#1173)
The current solution for titles on input boxes is not obvious to the
user as support varies between bridges. This commit adds an button to
all input boxes with titles in order to make it clear to the user that
additional information is available.
2019-10-31 21:09:44 +01:00
Roliga c8d5c85c76 formats: Add getMimeType() function (#1299)
Allows getting the expected MIME type of the format's output. A
corresponding MIME_TYPE constant is also defined in FormatAbstract for
the format implementations to overwrite.
2019-10-31 19:00:12 +01:00
somini d1e4bd7285 [YahtzeeDevDiary] Add new bridge (#1297) 2019-10-31 18:55:08 +01:00
LogMANOriginal 1022b5fdf9
core: Add an option to suppress error reporting (#1179)
Error reporting currently takes place for each error. This can result
in many error messages if a server has connectivity issues (i.e. when
it re-connects to the internet every 24 hours).

This commit adds a new option to the configuration file to define the
number of error reports to suppress before returning an error message
to the user.

Error reports are cached and therefore automatically purged after 24
hours. A successful bridge request does **not** clear the error count
as sporadic issues can be the result of actual problems on the server.

The implementation currently makes no assumption on the type of error,
which means it also suppresses bridge errors in debug mode. The default
value is, however, set to 1 which means all errors are reported.

References #994
2019-10-31 18:49:45 +01:00
LogMANOriginal e8536ac1b2
core: Add an option to return errors in different formats (#1071)
Bridge errors are currently included as part of the feed to
notify users about erroneous bridges (before that, bridges
silently failed).

This solution, however, can produce a high load of error
messages if servers are down (see #994 for more details).

Admins may also not want to include error messages in feeds
in order to keep those kind of problems away from users or
simply to silently fail by choice.

This commit adds a new configuration section "error" with
one option "output" which can be set to following values:

"feed": To include error messages in the feed (default)
"http": To return a HTTP header for each error
"none": To disable error reporting

Note that errors are always logged to 'error.log' independent
of the settings above.

Closes #1066
2019-10-31 18:40:51 +01:00
Lyra a0afe36d56 [DownDetectorBridge] Add per-website status fetch. Note that this only fetches the last downtime, as this is the only thing that the API provides. Moreover, the site uses a different ID for every company for every country, resulting in a very large array 2019-10-29 23:14:51 +01:00
Lyra 0b80f9d61c [DownDetectorBridge] Add bridge for DownDetector, and all local variants. Fixes
#1339.
2019-10-29 19:11:28 +01:00
somini 424075981f [EsquerdaNetBridge] Add new Bridge (#1296) 2019-10-29 18:58:12 +01:00
logmanoriginal c334df91ec composer: Add all details to the composer file
The composer file currently lacks a lot of details, especially the
"name" and "description", but also "require-dev" and "suggest" info.

This commit adds many more details to the composer file and updates
composer.lock for this repository. Technically the project is ready
to be shipped as composer package.
2019-10-28 20:01:19 +01:00
Dominik Thiemermann f2346fb33e [RevolutBridge] Add new bridge (#1321)
* [RevolutBridge] Add new bridge
2019-10-28 19:49:01 +01:00
Matt DeMoss 8a21fd1476 [BloombergBridge] Remove after site redesign and paywall. (#1238) 2019-10-28 19:27:56 +01:00
somini 2ac44172ac Facebook: Clarify Facebook bridges (#1221)
* Clarify Facebook bridges status

Distinguish between both Facebook bridges by their title.
This preserves all existing URLs.

* Update all URLs to secure HTTPS versions.
* Configure author name abbreviation
* Improve feed names

Use the correct feed name on each bridge.
Make sure the feed names don't repeat the "Facebook" name.
2019-10-28 19:01:04 +01:00
Roliga 4c78721f03 [ParameterValidator] Ensure context has all user provided parameters (#1211)
* [ParameterValidator] Ensure context has all fields

Previously if a bridge had a set of parameters like:

const PARAMETERS = array(
    'ContextA' => array(
        'Param1' => array(
            'name' => 'Param1',
            'required' => true
        )
    ),
    'ContextB' => array(
        'Param1' => array(
            'name' => 'Param1',
            'required' => true
        ),
        'Param2' => array(
            'name' => 'Param2',
            'required' => true
        )
    )
)

and a query specifying both Param1 and Param2 was provided a 'Mixed
context parameters' error would be returned. This change ensures
ContextA in the above example would not be considered a relevant context.
2019-10-28 17:50:55 +01:00
Christian Archer 04be85996d [BastaBridge] Fix PHP 7.4 crash (#1323)
* Inline the function
2019-10-24 21:57:14 +02:00
Joseph 59be6bded2 [GoogleSearchBridge] Replace 'div[id=ires]' with 'div[id=res]' (#1329) 2019-10-16 21:44:41 +02:00
Joseph 46873e14fe [GoogleSearchBridge] Use getURI() to build URLs (#1330)
* [GoogleSearchBridge] Use getURI() to build URLs
2019-10-16 21:44:28 +02:00
Joseph 0f01cc97a4 [StoriesIGBridge] Add timestamp to feed items (#1331) 2019-10-16 21:44:01 +02:00
Joseph a70e00a76d [SuperbWallpapersBridge] Delete bridge (#1336) 2019-10-16 21:43:38 +02:00
Joseph f0260c62c3 [StoriesIGBridge] Use getName() to create custom feed titles (#1332)
* [StoriesIGBridge] Use getName()
2019-10-16 21:41:47 +02:00
Roliga fc5a1526ca [BandcampBridge] Add band and album feeds (#1317)
* [BandcampBridge] Add band and artist feeds

This can return a limited number of the most recent releases by a band,
or a single release/album. Each release may be given a unique article ID
depending on its track list with the "Releases, new one when track track
changes" option, which should make them show up as new articles when
tracks are added or removed. Releases may also be split up to individual
articles for each track with the "Individual tracks" option.

This uses and undocumented API from the Bandcamp Android app. It's much
faster than loading and parsing the website HTML, and seems to fail less
often with more relaxed rate limits. It's still far from perfect in that
regard though.

The "Individual tracks" option generates requests for each individual
track so that can quickly run into rate limits.

The "Individual tracks" option also has a quirk where tracks released
under e.g. a music label will have their artist set to the label instead
of the actual artist of the track. This is a limitation of the API.
2019-10-16 21:37:25 +02:00
Joseph 4c0e234479 [Bridges] Use HTTPS (#1337)
* [Rule34pahealBridge] Use HTTPS
* [KonachanBridge] Use HTTPS
* [Rule34Bridge] Use HTTPS
* [SafebooruBridge] Use HTTPS
* [TbibBridge] Use HTTPS
* [XbooruBridge] Use HTTPS
* [ScmbBridge] Use HTTPS
* [ReporterreBridge] Use HTTPS
* [BastaBridge] Use HTTPS
* [NiceMatinBridge] Use HTTPS
* [ScoopItBridge] Use HTTPS
* [TheCodingLoveBridge] Use HTTPS
* [Shimmie2Bridge] Use HTTPS
* [HDWallpapersBridge] Use HTTPS
* [GiphyBridge] Use HTTPS
* [PickyWallpapersBridge] Use HTTPS
* [ParuVenduImmoBridge] Use HTTPS
* [ElsevierBridge] Use HTTPS
* [CastorusBridge] Use HTTPS
* [CollegeDeFranceBridge] Use HTTPS
* [MangareaderBridge] Use HTTPS
2019-10-16 21:34:28 +02:00
somini 0eab63d728 Update Facebook URL detection (#1334)
* add detectParameters to FacebookBridge.php
2019-10-16 21:32:29 +02:00
floviolleau b0884e9158 [VieDeMerdeBridge] Add new bridge for quotes from Vie de Merde (#1313)
* Add new bridge for quotes from Vie de Merde
2019-10-03 22:36:08 +02:00
Nicolas Delsaux b4581418d4 [PlantUMLBridge] Added bridge for PlantUML. Fixes #1191
* Fixes #1191 by implementing the RSS feed of PlantUML releases
2019-10-03 22:30:22 +02:00
sysadminstory af1566f40d [ZoneTelechargementBridge] URL and name change (#1302)
Annuaire Telechargement has change name again to go back to Annuaire
Telechargement. Fixes #1279
2019-10-03 22:27:10 +02:00
sysadminstory 529e0d0cca [ExtremeDownloadBridge] Update Website URL (#1303)
Website URL was changed.
2019-10-03 22:25:56 +02:00
Paróczai Olivér a3532804ac [Readme] Small grammar fixes (#1312) 2019-10-03 22:25:05 +02:00
Anchit Bajaj 8c19146d29 [ListverseBridge - add new bridge (#1305) 2019-10-03 22:24:14 +02:00
Anchit Bajaj 2a3d5865ad [FreeCodeCampBridge] - rss feed for FreeCodeCamp (#1311) 2019-10-03 22:23:14 +02:00
Lyra 4d36c9dc30 Merge branch 'master' of github.com:RSS-Bridge/rss-bridge 2019-10-03 22:14:33 +02:00
Lyra a2e47a88c3 [InstagramBridge] Add option to get direct links 2019-10-03 22:14:21 +02:00
Anchit Bajaj b09f50853f [ViceBridge] - RSS feed for Vice Publications. (#1310)
* [ViceBridge] - RSS feed for Vice Publications.
2019-10-03 22:02:30 +02:00
lukasklinger 9b5bf565b3 [N26Bridge] Updated bridge to reflect changes on N26 blog (#1295)
N26 made some changes to their blog, this commit fixes the N26Bridge
2019-10-03 21:58:57 +02:00
Lyra 5cc956367f [core] Fix travis 2019-10-03 21:46:49 +02:00
Lyra 548e28249b [ThePirateBayBridge] Remove nested function 2019-10-03 21:46:24 +02:00
Lyra 684c69b0cd [Releases3DSBridge] Remove nested functions 2019-10-03 21:46:09 +02:00
Lyra 3dae4e0801 [JapanExpoBridge] Remove nested function 2019-10-03 21:45:51 +02:00
Lyra 4622d9be1e [ReadComicsBridge] Deleted bridge since website no longer exists 2019-10-03 21:41:22 +02:00
Nicolas Delsaux 76183dcd44 [GQMagazineBridge] Fix article body detection again (Fixes #1280) 2019-10-03 21:26:41 +02:00
Eugene Molotov 50b234d893 [VkBridge] Photo and timestamp fixes (#1287)
* [VkBridge] Correct parsing of photos, fix timestamp for old posts
2019-09-16 21:30:27 +02:00
Eugene Molotov af48f36fd2 [VkBridge] Switch maintainer (#1288) 2019-09-16 21:29:45 +02:00
Eugene Molotov 7f6ca23e8f [PikabuBridge] Preserve links (#1286)
* [PikabuBridge] Preserve links
2019-09-16 21:28:41 +02:00
oratosquilla-oratoria 1daef22a3d [NFLRUSBridge] Add new bridge (#1285)
* [NFLRUSBridge] Add new bridge
2019-09-16 21:27:01 +02:00
killruana c694810d9a [MediapartBridge] Fix article parsing
* Only process article item, fix issue #1292
2019-09-16 21:26:19 +02:00
ORelio f12f6a2dba [DarkReading] Add DarkReading Bridge (#1289) 2019-09-16 21:25:28 +02:00
Lyra b1be45df6c
[Configuration] Bump version to dev.2019-09-12 2019-09-12 17:09:30 +02:00
Lyra b4f393a5cc
[Configuration] Bump version to 2019-07-06 2019-09-12 17:08:15 +02:00
Lyra 29126ebe29
[README] Update list of contributors 2019-09-12 17:07:04 +02:00
triatic 50c971d545 [TwitterBridge] Enable cookies with curl (#1245)
* [TwitterBridge] Enable cookies with curl

Enable cookies in curl, or fall back to `file_get_contents` if in CLI mode with no curl root certificates.
2019-09-12 16:14:48 +02:00
Lyra 7aba7992aa [InstagramBridge] Remove condition that forces cache ignoring 2019-09-11 19:28:46 +02:00
Lyra 48ebed7b38 [InstagramBridge] Fix Instagram stories and user id finding. 2019-09-11 19:08:12 +02:00
Lyra ccef6b95ad [InstagramBridge] Attempt to fix the queries in order to bypass rate limits 2019-09-10 14:37:50 +02:00
Antoine Turmel dd5da99a30 [StoriesIGBridge] New bridge (#1187)
* Create StoriesIGBridge.php
2019-09-07 18:43:06 +02:00
Joseph 2ff27b92ff [DailymotionBridge] Use API for playlist and user account feeds (#1217) 2019-09-07 18:42:45 +02:00
Joseph b47189921f [CuriousCatBridge] Add new bridge (#1216)
* Create CuriousCatBridge.php
2019-09-07 18:37:30 +02:00
floviolleau f1d3e8c9c9 [AtmoNouvelleAquitaineBridge] Add new bridge for air quality in Bordeaux (#1229)
* Add new bridge for air quality in Bordeaux
2019-09-07 18:36:55 +02:00
triatic 53fbd2a5a0 [FacebookBridge] Prevent sending empty header (#1239)
* [FacebookBridge] Prevent sending empty header

When running in CLI mode, `getEnv('HTTP_ACCEPT_LANGUAGE')` returns `false`. In that case, don't send the `Accept-Language` header.
2019-09-07 18:32:06 +02:00
ORelio 3254a4d7bc [WIRED] Add WIRED Bridge (#1244)
* [WIRED] Add WIRED Bridge
2019-09-07 18:31:19 +02:00
Roliga 52d2d21da5 [TwitchBridge] Add new bridge (#1253)
* [TwitchBridge] Add new bridge
2019-09-07 18:27:44 +02:00
Roliga abb74f056c [PatreonBridge] Add new bridge (#1254)
* [PatreonBridge] Add new bridge

* [PatreonBridge] Add UID to articles

Patreon changes post URLs when the post title is updated, so set a UID
based on the post ID instead.
2019-09-07 18:26:58 +02:00
dawidsowa 25548b6757 [Rule34pahealBridge] Fix thumbnail uri (#1278) 2019-09-07 18:26:08 +02:00
sysadminstory cfe433e9e2 [AutoJMBridge] Fix the bridge to follow website changes (#1255)
The Website changed in two way :
- The filter about availability disappeared (and this leads to a
  parameters change, which will break existing bridges, sorry)
- Some HTML change
2019-09-06 10:52:58 +02:00
Nicolas Delsaux 0dfc4ea2c5 [GQMagazineBridge] Adapt to changes, fixes #1280 2019-09-06 10:51:13 +02:00
Lyra 38960df180 [ThePirateBayBridge] Fix PHPCS code violations 2019-09-06 10:55:15 +02:00
Eugene Molotov b440a6fdc6 [PikabuBridge] Added filtering by user (#1266) 2019-08-28 16:29:49 +02:00
somini 48d0385653 [core] Fix double XML encoding on Atom feed title (#1247) 2019-08-28 16:29:13 +02:00
Roliga b68c0e0df8 [PirateCommunityBridge] Add new bridge (#1252)
* [PirateCommunityBridge] Add new bridge
2019-08-28 16:28:39 +02:00
Anchit Bajaj f27b267614 [GuardianBridge] - New bridge for the Guardian (#1249)
* [GuardianBridge] - New bridge for the Guardian
2019-08-28 16:27:45 +02:00
Mitsu 8bff63d9c6
[ThePirateBay] URI fix, add magnet link 2019-08-27 01:18:43 +02:00
Mitsu 2b4a030158
[ThePirateBay] switch back TLD to .org
And the "whack-a-mole" game continues
2019-08-27 00:55:36 +02:00
Rudolf M. Schreier 6a99904e64 [DanbooruBridge] Decode href of HTML element to avoid double escaping. (#1262)
Directly accessing ...->href resulted in a string that contained '&amp;'
instead of '&'. This was later escaped again to '&amp;amp;' in some
formats (e.g. Atom).
2019-08-26 14:26:19 +02:00
sysadminstory f3c687604f [DealabsBridge] Follow website change (#1256)
A minor website change broke the Bridge. This commit fix it
2019-08-26 14:25:47 +02:00
Lyra a86a94555d [LeBonCoinBridge] Submit user agent to LBC to get results. 2019-08-26 14:22:58 +02:00
Anchit Bajaj acc0787b00 [IGNBridge] - New bridge for IGN (#1233)
* [IGNBridge]: New Bridge for IGN
2019-07-31 14:26:43 +02:00
johnnygroovy c8992650a1 [DavesTrailerPageBridge] Add new bridge (#1246) 2019-07-31 14:17:34 +02:00
Anchit Bajaj f9f511a849 [NYTBridge] : New bridge for the new york times (#1235) 2019-07-29 12:15:08 +02:00
somini 990719d614 [FabriceBellard]: New Bridge (#1220)
* [FabriceBellard]: New Bridge
2019-07-29 12:12:55 +02:00
triatic b6be18d585 [contents] Respect passed headers for file_get_contents() (#1234)
* [contents] Respect passed headers for file_get_contents()
2019-07-29 12:05:13 +02:00
Roliga cf525c964a [WIP][FurAffinityBridge] Add new bridge (#1083)
* [FurAffinityBridge] Add new bridge
2019-07-26 11:02:58 +02:00
Antoine Cadoret 52a4f0860c [LaCentraleBridge] Add new bridge (#1201)
* [LaCentraleBridge] Introduce new bridge
2019-07-26 11:00:55 +02:00
triatic 21b27a1042 [FacebookBridge] Remove relative date from content (#1212)
Remove relative date from content, as well as the separator after it.

As mentioned in #1188.
2019-07-26 10:56:34 +02:00
Léo Maradan 2eee535171 CNET France Bridge (#1214)
CNET France News but with filters on title or url
2019-07-26 10:53:09 +02:00
Anchit Bajaj da51fc065f [EngadgetBridge] New bridge for Engadget (#1215)
* [EngadgetBridge] New bridge for Engadget
2019-07-26 10:51:20 +02:00
Joseph e032705c9a [HaveIBeenPwnedBridge] Add item limit parameter, set default limit to 20 (#1219)
* Add `item_limit` parameter to allow user to control number of item returned by bridge. Suggested by @triatic and @somini (code).
2019-07-26 10:47:20 +02:00
Joseph be27bc9250 Fix malformed URLs (#1222)
Removes 'self::URI' from processUpload() which was creating malformed URLs. Relative URLs are handled by defaultLinkTo() making 'self::URI' unnecessary.
2019-07-26 10:43:18 +02:00
Albirew 75edc1b2b7 [NovelUpdatesBridge] now in https (#1228) 2019-07-26 10:42:41 +02:00
Albirew c9ea53806d [HentaiHavenBridge] now in https (#1227) 2019-07-26 10:42:19 +02:00
triatic 2bb9480555 [TwitterBridge] Get cookies before sending request (#1232)
* [TwitterBridge] Get cookies before sending request

Twitter now requires cookies to be set before requesting a page. This will fetch the cookies and send them to `getSimpleHTMLDOM()`.

* Formatting fixes
2019-07-26 10:36:59 +02:00
Corentin Garcia eb942bc498 [UnsplashBridge] Fix bridge (fix issue #965) (#1208) 2019-07-16 16:50:14 +02:00
logmanoriginal 5a0ea423c4 [Configuration] Bump version to dev.2019-07-06 2019-07-06 12:35:36 +02:00
logmanoriginal 2120cc42fb [Configuration] Bump version to 2019-07-06 2019-07-06 12:34:42 +02:00
logmanoriginal 5067501661 [README] Update list of contributors 2019-07-06 12:34:42 +02:00
LogMANOriginal aea8484ccc
[FicbookBridge] Add new bridge (#1185) 2019-07-06 12:29:36 +02:00
logmanoriginal 6b9394dc78 [DemonoidBridge] Remove bridge
The public service demonoid.pw is no longer available and is
currently being rebuild under demonoid.info which hides torrents
behind a login wall. As this is not supported by RSS-Bridge, the
bridge will be removed.

Find more details on Reddit:
https://www.reddit.com/r/Demonoid/
2019-07-06 12:25:23 +02:00
logmanoriginal 4b51d42b8c cache: Keep subfolders in the repository
References #1200
2019-07-06 12:12:59 +02:00
Joseph d3fbf0d872 Fix bridge description (#1207) 2019-07-06 11:59:55 +02:00
Joseph 41a8eb74a1 [PinterestBridge] Remove search (#1206)
* Remove getSearchResults()
* Remove ''From search' from PARAMETERS array
* Update getURI() and getName()
* Update collectData()
* Add '.rss' to URL in `collectData` instead of in `getURI`
2019-07-06 11:57:48 +02:00
Joseph 7e6c58b67a [HaveIBeenPwnedBridge] Display breach type (#1203)
* Extract breach types for each data breach
* Add paragraph tag
2019-07-06 11:55:31 +02:00
triatic a31e518a07 [TelegramBridge] Fix forwarded videos (#1202)
Videos forwarded from other channels use a slightly different format, This fixes it.
2019-07-06 11:52:56 +02:00
logmanoriginal 50162f52b6 [XenForoBridge] Fix minor issues with CSS selectors 2019-07-03 19:34:43 +02:00
logmanoriginal c0edf6e424 [ShanaprojectBridge] Add filter options
- Filter by minimum number of episodes
- Filter by minimum number of total episodes
- Filter by banner image
2019-07-03 19:34:43 +02:00
logmanoriginal 2ea8d73ac1 [ShanaprojectBridge] Return url to current season 2019-07-02 20:46:38 +02:00
logmanoriginal 465cd8c768 [ShanaprojectBridge] Add support for https and cleanup 2019-07-02 20:45:31 +02:00
logmanoriginal 73f4bc078e [CastorusBridge] Fix broken activity selector 2019-06-28 20:31:49 +02:00
Nicolas Delsaux 1add201d3b [WorldOfTanksBridge] Fix bridge (#1197)
* Fix #1196 by better protecting page
2019-06-28 19:32:26 +02:00
Nicolas Delsaux 09113c2594 [GQMagazineBridge] Fix bridge (#1195)
* Fix bridge by changing the way the articles are loaded AND their titles are found
2019-06-28 19:29:32 +02:00
Joseph c39e642877 [HaveIBeenPwnedBridge] Convert HTML entities to characters (#1198) 2019-06-28 16:08:56 +02:00
Joseph e2460ead18 [InternetArchiveBridge] Add new bridge (#1186) 2019-06-28 15:45:27 +02:00
logmanoriginal 60c1339612 [InstructablesBridge] Fix after layout changes 2019-06-27 21:05:50 +02:00
logmanoriginal d324aa5da1 [InstructablesBridge] Update available categories 2019-06-27 20:29:21 +02:00
logmanoriginal 6f24987601 [InstructablesBridge] Fix listCategories() to work with new layout 2019-06-27 20:28:23 +02:00
logmanoriginal 54fb29d443 [InstructablesBridge] Add support for HTTPS 2019-06-27 20:16:53 +02:00
Joseph ebe463dd08 [TelegramBridge] Set 'username' parameter as required (#1192) 2019-06-27 20:03:18 +02:00
logmanoriginal 987f42d6d4 logo: Add logo to the project
References #1087
2019-06-25 18:42:11 +02:00
logmanoriginal fa8253c8bf [GiteaBridge] Add new bridge
Gitea is a fork of Gogs and therefore shares most of its features
except for releases.
2019-06-23 09:21:00 +02:00
logmanoriginal e4444e6432 [GogsBridge] Add new bridge 2019-06-23 09:21:00 +02:00
triatic 3769850ba3 [TelegramBridge] Fix entries for "media too big" (#1184)
When a large video is posted, "Media is too big" appears in web preview. This adds code to detect this and offer a link.
2019-06-23 08:54:52 +02:00
LogMANOriginal 89e3da0b6f
[IndeedBridge] Add new bridge (#1166)
Implements a bridge for
https://www.indeed.com/ (or any of the local variants)

Features:
- Takes a company name and returns a list of reviews and comments
- Limit the maximum number of items to return (default: 20)
- No upper limit on the number of items to return
- Search by language code (45 options)
- Supports detectParameters for any supported URL
2019-06-22 18:50:06 +02:00
logmanoriginal 99d4571c6b core: Make RSS-Bridge more usable via mobile devices
Adds styles for display sizes smaller than 768px where
elements are currently hardly usable. Note that RSS-Bridge
is not designed for mobile use, but some users may want
to try things on their mobile phone before using it in
real life applications.

Resolves #796
2019-06-22 18:46:37 +02:00
triatic 69acc6228a [TelegramBridge] Populate author (#1183) 2019-06-22 18:45:15 +02:00
triatic 5e2f0fb626 [TelegramBridge] Prevent double encoding entities (#1182) 2019-06-22 18:44:25 +02:00
triatic 372461b1a3 [TelegramBridge] Fix timestamp for videos (#1181) 2019-06-22 18:34:02 +02:00
logmanoriginal 1591e18027 core: Add context hinting for new feeds
RSS-Bridge currently has to guess the queried context from the data
provided by the user. This, however, can cause issues for bridges
that have multiple contexts with conflicting parameters (i.e. none).

This commit adds context hinting to queries via '&context=<context>'
which can be omitted in which case the context is determined as before.
2019-06-21 19:12:29 +02:00
husimo e2bca5bb05 [MastodonBridge] Add new bridge (#1178) 2019-06-21 17:30:34 +02:00
logmanoriginal 7926ffad73 [KununuBridge] Improve feed contents
- Add support for ratings
- Add support for benefits
- Fix broken timestamp
2019-06-21 00:00:44 +02:00
logmanoriginal 7ff97c0c7b [HtmlFormat] Dynamically build buttons for other feed formats
Adding or removing feed formats from the "formats/" directory
currently has no effect on the buttons shown in the HTML format.
This can cause errors if users press one of the buttons for a
format that is no longer available on the server.

This commit changes the behavior to dynamically add buttons based
on the available formats. Syndication feeds, however, are no longer
supported as they require knowledge about the content type, which
is not known without further changes to the formats API (may be
added later if there is a demand).

Closes #942
2019-06-19 23:13:37 +02:00
Joseph 1989252608 [TelegramBridge] Add new bridge (#1175) 2019-06-19 22:40:56 +02:00
LogMANOriginal 91e73b00b5
[NationalGeographicBridge] Add new bridge (#1065)
Closes #1029
2019-06-18 22:57:42 +02:00
LogMANOriginal 5c6c79baf4
[VimeoBridge] Add new bridge (#933)
Closes #932
2019-06-18 22:50:31 +02:00
Joseph 99d1343045 [SplCenterBridge] Add new bridge (#1177) 2019-06-18 22:18:52 +02:00
logmanoriginal 14e6dbb645 [ListActionTest] Fix broken test 2019-06-18 19:21:28 +02:00
logmanoriginal fc8421ed50 format: Refactor format factory to non-static class
The format factory can be based on the abstract factory class if it
wasn't static. This allows for higher abstraction and makes future
extensions possible. Also, not all parts of RSS-Bridge need to work
on the same instance of the factory.

References #1001
2019-06-18 19:15:20 +02:00
logmanoriginal 2460b67886 cache: Refactor cache factory to non-static class
The cache factory can be based on the abstract factory class if it
wasn't static. This allows for higher abstraction and makes future
extensions possible. Also, not all parts of RSS-Bridge need to work
on the same instance of the factory.

References #1001
2019-06-18 19:04:19 +02:00
logmanoriginal 705b9daa0b bridge: Refactor bridge factory to non-static class
The bridge factory can be based on the abstract factory class if it
wasn't static. This allows for higher abstraction and makes future
extensions possible. Also, not all parts of RSS-Bridge need to work
on the same instance of the bridge factory.

References #1001
2019-06-18 18:55:32 +02:00
logmanoriginal 1ada9c26f8 format: Sanitize format name in the format factory
RSS-Bridge currently sanitizes the format name only for the display
action, which can cause problems if other actions depend on formats
as well.

It is therefore better to do sanitization in the factory class for
formats. Additionally, formats should not require a perfect match,
so 'Atom' and 'aToM' make no difference. This will also allow users
to define formats in their own style (i.e. only lowercase via CLI).

References #1001
2019-06-18 18:36:16 +02:00
Corentin Garcia 55e1703741 [EliteDangerousGalnetBridge] Remove duplicate items (#1167) 2019-06-16 20:35:23 +02:00
Tobias Alexander Franke 849eaeb50e [SteamCommunityBridge] Add Workshop category (#1172) 2019-06-16 20:21:48 +02:00
Thibault Couraud aeca4cfd60 [BAEBridge] Use defaultLinkTo rather than str_replace (#1168) 2019-06-16 19:40:21 +02:00
Thibault Couraud 686f21bc50 [FindACrew] Improve bridge results (#1120) 2019-06-16 19:35:43 +02:00
LogMANOriginal 8dd8be9694
[.gitattributes] Keep files in export for Heroku
Heroku requires the file `app.json` as well as the composer files
`composer.json` and `composer.lock` to deploy a service. Deploy
doesn't work if these files are ignored during export (because of
the way this service deploys projects).

This commit adds comments to .gitattributes to prevent this issue
from re-appearing in the future. All affected lines are commented
out.

Also added some spacing for better readability.

References #1165
2019-06-16 19:15:28 +02:00
logmanoriginal dfa9c651cd [BridgeList] Change placeholder message in the search bar
The search bar should indicate that searching by URL is
supported.

References #1099
2019-06-13 19:55:10 +02:00
logmanoriginal 6d6d6037a3 [GithubIssueBridge] Don't return error messages in detectParameters()
detectParameters() is called in a loop for all bridges on a URL, thus
if a bridge returns an error message, the output messages get mixed
up and all detect operations fail.

This seems to be a limitation of the detect function for now.
2019-06-13 19:49:54 +02:00
Joseph 2559dbbf49 [BrutBridge] Create custom feed name for each category and edition (#1164) 2019-06-13 19:13:02 +02:00
logmanoriginal de53120843 [SakugabooruBridge] Remove bridge
The target server for this bridge is no longer reachable and
there doesn't seem to be any attempt to get it back online.
2019-06-12 20:22:53 +02:00
logmanoriginal b1b7e4edce [DollbooruBridge] Remove bridge
The target site for this bridge has been down for at least a year
now and there doesn't seem to be any attempt to get it back up.
Their twitter account is also silent since 2012, so no harm
removing this bridge.

https://twitter.com/dollbooru?lang=en
2019-06-12 20:11:34 +02:00
logmanoriginal b27487ace0 [TwitterBridge] Fix detection of retweets on lists
References #1161
2019-06-12 18:27:35 +02:00
logmanoriginal d005acca83 [TwitterBridge] Add extensive description to keyword search query
References #1163
2019-06-11 21:53:22 +02:00
LogMANOriginal 93de8c239b
[README] Remove GooglePlus from supported sites 2019-06-10 15:40:57 +02:00
logmanoriginal 75b0213684 [GithubIssueBridge] Add support for detect action
References #1100
2019-06-10 15:32:57 +02:00
Eugene Molotov f76a23f0a5 [YoutubeBridge] Add playlist caching (#1162) 2019-06-10 15:31:35 +02:00
logmanoriginal e4e04a7865 [GithubIssueBridge] Fix broken feed item URLs
References #1100
2019-06-10 00:02:13 +02:00
logmanoriginal da339fd5cc [GithubIssueBridge] Include issue author comment in the feed
- Add function to build an URL to the GitHub issue comment
- Change scope of internal functions from protected to private
- Use IDs instead of classes as comment selectors, to include the
issue author in the output feed.

References #1100
2019-06-09 20:39:45 +02:00
logmanoriginal ba116d9ab6 [GithubIssueBridge] Fix bridge after DOM changes 2019-06-09 19:57:48 +02:00
logmanoriginal ea08445946 [GlassdoorBridge] Fix broken bridge 2019-06-09 19:35:53 +02:00
logmanoriginal ade09b2aad [XenForoBridge] Fix broken bridge 2019-06-09 19:35:53 +02:00
logmanoriginal 28d46b6721 [ShanaprojectBridge] Fix broken bridge 2019-06-09 19:35:46 +02:00
logmanoriginal 1efb7c7bce [DesoutterBridge] Fix bridge after DOM changes 2019-06-09 19:01:54 +02:00
Joseph d34411137f [TwitterBridge] Display all images from a tweet (#1160) 2019-06-09 17:24:40 +02:00
logmanoriginal 70542686bb [contents] Fix parsing of incomplete headers
Response headers may contain fields with no values.

Example:
  "Referrer-Policy: "

In this case the current implementation of explode() results in an
error because there is no content after ": ". Changing the delimiter
to ":" and trimming the value manually fixes that issue.
2019-06-09 17:18:08 +02:00
LogMANOriginal edf10be93a
[README] Change color for Guix release to blue
This prevents confusion with the build status for Travis-CI and Docker
2019-06-08 20:36:59 +02:00
LogMANOriginal a725fdd315
[README] Add logos to badges where applicable 2019-06-08 20:27:41 +02:00
logmanoriginal 84ba0c4a9e [Configuration] Bump version to dev.2019-06-08 2019-06-08 20:12:04 +02:00
168 changed files with 21018 additions and 1877 deletions

26
.gitattributes vendored
View File

@ -22,18 +22,24 @@
*.RTF diff=astextplain
# Ignore files in git archive (i.e. GitHub release builds)
## Docker
Dockerfile export-ignore
.dockerignore export-ignore
## Travis
.travis.yml export-ignore
## GitHub
.github/ export-ignore
## Git
.gitattributes export-ignore
.gitignore export-ignore
## Scalingo
scalingo.json export-ignore
## RSS-Bridge
phpunit.xml export-ignore
phpcs.xml export-ignore
@ -42,8 +48,22 @@ tests/ export-ignore
cache/.gitkeep export-ignore
bridges/DemoBridge.php export-ignore
bridges/FeedExpanderExampleBridge.php export-ignore
## Composer
composer.json export-ignore
composer.lock export-ignore
#
# Keep the following lines commented out. Heroku does
# not function if the composer files are ignored during
# export. For more information see
# https://github.com/rss-bridge/rss-bridge/issues/1165
#
# composer.json export-ignore
# composer.lock export-ignore
## Heroku
app.json export-ignore
#
# Keep the following line commented out. Heroku does
# not function if app.json is ignored during export.
# For more information see
# https://github.com/rss-bridge/rss-bridge/issues/1165
#
# app.json export-ignore

View File

@ -6,6 +6,8 @@ RUN mv "$PHP_INI_DIR/php.ini-production" "$PHP_INI_DIR/php.ini" \
&& apt-get --yes update && apt-get --yes install libxml2-dev \
&& docker-php-ext-install -j$(nproc) simplexml \
&& sed -ri -e 's!/var/www/html!${APACHE_DOCUMENT_ROOT}!g' /etc/apache2/sites-available/*.conf \
&& sed -ri -e 's!/var/www/!${APACHE_DOCUMENT_ROOT}!g' /etc/apache2/apache2.conf /etc/apache2/conf-available/*.conf
&& sed -ri -e 's!/var/www/!${APACHE_DOCUMENT_ROOT}!g' /etc/apache2/apache2.conf /etc/apache2/conf-available/*.conf \
&& sed -ri -e 's/(MinProtocol\s*=\s*)TLSv1\.2/\1None/' /etc/ssl/openssl.cnf \
&& sed -ri -e 's/(CipherString\s*=\s*DEFAULT)@SECLEVEL=2/\1/' /etc/ssl/openssl.cnf
COPY --chown=www-data:www-data ./ /app/

235
README.md
View File

@ -1,8 +1,8 @@
rss-bridge
![RSS-Bridge](static/logo_600px.png)
===
[![LICENSE](https://img.shields.io/badge/license-UNLICENSE-blue.svg)](UNLICENSE) [![GitHub release](https://img.shields.io/github/release/rss-bridge/rss-bridge.svg)](https://github.com/rss-bridge/rss-bridge/releases/latest) [![Debian Release](https://img.shields.io/badge/dynamic/json.svg?label=debian%20release&url=https%3A%2F%2Fsources.debian.org%2Fapi%2Fsrc%2Frss-bridge%2F&query=%24.versions%5B0%5D.version&colorB=blue)](https://tracker.debian.org/pkg/rss-bridge) [![Guix Release](https://img.shields.io/badge/guix%20release-unknown-light--gray.svg)](https://www.gnu.org/software/guix/packages/R/) [![Build Status](https://travis-ci.org/RSS-Bridge/rss-bridge.svg?branch=master)](https://travis-ci.org/RSS-Bridge/rss-bridge) [![Docker Build Status](https://img.shields.io/docker/build/rssbridge/rss-bridge.svg)](https://hub.docker.com/r/rssbridge/rss-bridge/)
[![LICENSE](https://img.shields.io/badge/license-UNLICENSE-blue.svg)](UNLICENSE) [![GitHub release](https://img.shields.io/github/release/rss-bridge/rss-bridge.svg?logo=github)](https://github.com/rss-bridge/rss-bridge/releases/latest) [![Debian Release](https://img.shields.io/badge/dynamic/json.svg?logo=debian&label=debian%20release&url=https%3A%2F%2Fsources.debian.org%2Fapi%2Fsrc%2Frss-bridge%2F&query=%24.versions%5B0%5D.version&colorB=blue)](https://tracker.debian.org/pkg/rss-bridge) [![Guix Release](https://img.shields.io/badge/guix%20release-unknown-blue.svg)](https://www.gnu.org/software/guix/packages/R/) [![Build Status](https://travis-ci.org/RSS-Bridge/rss-bridge.svg?branch=master)](https://travis-ci.org/RSS-Bridge/rss-bridge) [![Docker Build Status](https://img.shields.io/docker/build/rssbridge/rss-bridge.svg?logo=docker)](https://hub.docker.com/r/rssbridge/rss-bridge/)
RSS-Bridge is a PHP project capable of generating RSS and Atom feeds for websites which don't have one. It can be used on webservers or as stand alone application in CLI mode.
RSS-Bridge is a PHP project capable of generating RSS and Atom feeds for websites that don't have one. It can be used on webservers or as a stand-alone application in CLI mode.
**Important**: RSS-Bridge is __not__ a feed reader or feed aggregator, but a tool to generate feeds that are consumed by feed readers and feed aggregators. Find a list of feed aggregators on [Wikipedia](https://en.wikipedia.org/wiki/Comparison_of_feed_aggregators).
@ -15,7 +15,6 @@ Supported sites/pages (examples)
* `DuckDuckGo`: Most recent results from [DuckDuckGo.com](https://duckduckgo.com/)
* `Facebook` : Returns the latest posts on a page or profile on [Facebook](https://facebook.com/)
* `FlickrExplore` : [Latest interesting images](http://www.flickr.com/explore) from Flickr
* `GooglePlus` : Most recent posts of user timeline
* `GoogleSearch` : Most recent results from Google Search
* `Identi.ca` : Identica user timeline (Should be compatible with other Pump.io instances)
* `Instagram`: Most recent photos from an Instagram user
@ -77,7 +76,7 @@ RSS-Bridge allows you to take full control over which bridges are displayed to t
Find more information on the [Wiki](https://github.com/RSS-Bridge/rss-bridge/wiki/Whitelisting)
**Notice**: By default RSS-Bridge will only show a small subset of bridges. Make sure to read up on [whitelisting](https://github.com/RSS-Bridge/rss-bridge/wiki/Whitelisting) to unlock the full potential of RSS-Bridge!
**Notice**: By default, RSS-Bridge will only show a small subset of bridges. Make sure to read up on [whitelisting](https://github.com/RSS-Bridge/rss-bridge/wiki/Whitelisting) to unlock the full potential of RSS-Bridge!
Deploy
===
@ -111,110 +110,128 @@ Use this script to generate the list automatically (using the GitHub API):
https://gist.github.com/LogMANOriginal/da00cd1e5f0ca31cef8e193509b17fd8
-->
* [16mhz](https://github.com/16mhz)
* [adamchainz](https://github.com/adamchainz)
* [Ahiles3005](https://github.com/Ahiles3005)
* [Albirew](https://github.com/Albirew)
* [aledeg](https://github.com/aledeg)
* [alex73](https://github.com/alex73)
* [alexAubin](https://github.com/alexAubin)
* [AmauryCarrade](https://github.com/AmauryCarrade)
* [ArthurHoaro](https://github.com/ArthurHoaro)
* [Astalaseven](https://github.com/Astalaseven)
* [Astyan-42](https://github.com/Astyan-42)
* [az5he6ch](https://github.com/az5he6ch)
* [azdkj532](https://github.com/azdkj532)
* [b1nj](https://github.com/b1nj)
* [benasse](https://github.com/benasse)
* [captn3m0](https://github.com/captn3m0)
* [chemel](https://github.com/chemel)
* [ckiw](https://github.com/ckiw)
* [cnlpete](https://github.com/cnlpete)
* [corenting](https://github.com/corenting)
* [couraudt](https://github.com/couraudt)
* [da2x](https://github.com/da2x)
* [Daiyousei](https://github.com/Daiyousei)
* [disk0x](https://github.com/disk0x)
* [DJCrashdummy](https://github.com/DJCrashdummy)
* [Djuuu](https://github.com/Djuuu)
* [DnAp](https://github.com/DnAp)
* [Draeli](https://github.com/Draeli)
* [Dreckiger-Dan](https://github.com/Dreckiger-Dan)
* [em92](https://github.com/em92)
* [eMerzh](https://github.com/eMerzh)
* [EtienneM](https://github.com/EtienneM)
* [fluffy-critter](https://github.com/fluffy-critter)
* [Frenzie](https://github.com/Frenzie)
* [fulmeek](https://github.com/fulmeek)
* [Ginko-Aloe](https://github.com/Ginko-Aloe)
* [Glandos](https://github.com/Glandos)
* [GregThib](https://github.com/GregThib)
* [griffaurel](https://github.com/griffaurel)
* [Grummfy](https://github.com/Grummfy)
* [hunhejj](https://github.com/hunhejj)
* [j0k3r](https://github.com/j0k3r)
* [JackNUMBER](https://github.com/JackNUMBER)
* [jdigilio](https://github.com/jdigilio)
* [JeremyRand](https://github.com/JeremyRand)
* [Jocker666z](https://github.com/Jocker666z)
* [killruana](https://github.com/killruana)
* [klimplant](https://github.com/klimplant)
* [kranack](https://github.com/kranack)
* [kraoc](https://github.com/kraoc)
* [l1n](https://github.com/l1n)
* [laBecasse](https://github.com/laBecasse)
* [lagaisse](https://github.com/lagaisse)
* [lalannev](https://github.com/lalannev)
* [ldidry](https://github.com/ldidry)
* [Limero](https://github.com/Limero)
* [LogMANOriginal](https://github.com/LogMANOriginal)
* [lorenzos](https://github.com/lorenzos)
* [m0zes](https://github.com/m0zes)
* [matthewseal](https://github.com/matthewseal)
* [mcbyte-it](https://github.com/mcbyte-it)
* [mdemoss](https://github.com/mdemoss)
* [melangue](https://github.com/melangue)
* [metaMMA](https://github.com/metaMMA)
* [mitsukarenai](https://github.com/mitsukarenai)
* [MonsieurPoutounours](https://github.com/MonsieurPoutounours)
* [mr-flibble](https://github.com/mr-flibble)
* [mro](https://github.com/mro)
* [mxmehl](https://github.com/mxmehl)
* [nel50n](https://github.com/nel50n)
* [niawag](https://github.com/niawag)
* [Nono-m0le](https://github.com/Nono-m0le)
* [ObsidianWitch](https://github.com/ObsidianWitch)
* [ORelio](https://github.com/ORelio)
* [PaulVayssiere](https://github.com/PaulVayssiere)
* [pellaeon](https://github.com/pellaeon)
* [Piranhaplant](https://github.com/Piranhaplant)
* [pit-fgfjiudghdf](https://github.com/pit-fgfjiudghdf)
* [pitchoule](https://github.com/pitchoule)
* [pmaziere](https://github.com/pmaziere)
* [Pofilo](https://github.com/Pofilo)
* [prysme01](https://github.com/prysme01)
* [quentinus95](https://github.com/quentinus95)
* [regisenguehard](https://github.com/regisenguehard)
* [Riduidel](https://github.com/Riduidel)
* [rogerdc](https://github.com/rogerdc)
* [Roliga](https://github.com/Roliga)
* [sebsauvage](https://github.com/sebsauvage)
* [somini](https://github.com/somini)
* [squeek502](https://github.com/squeek502)
* [Strubbl](https://github.com/Strubbl)
* [sublimz](https://github.com/sublimz)
* [sysadminstory](https://github.com/sysadminstory)
* [tameroski](https://github.com/tameroski)
* [teromene](https://github.com/teromene)
* [thefranke](https://github.com/thefranke)
* [TheRadialActive](https://github.com/TheRadialActive)
* [triatic](https://github.com/triatic)
* [VerifiedJoseph](https://github.com/VerifiedJoseph)
* [WalterBarrett](https://github.com/WalterBarrett)
* [wtuuju](https://github.com/wtuuju)
* [xurxof](https://github.com/xurxof)
* [yardenac](https://github.com/yardenac)
* [ZeNairolf](https://github.com/ZeNairolf)
* [16mhz](https://github.com/16mhz)
* [adamchainz](https://github.com/adamchainz)
* [Ahiles3005](https://github.com/Ahiles3005)
* [Albirew](https://github.com/Albirew)
* [aledeg](https://github.com/aledeg)
* [alex73](https://github.com/alex73)
* [alexAubin](https://github.com/alexAubin)
* [AmauryCarrade](https://github.com/AmauryCarrade)
* [AntoineTurmel](https://github.com/AntoineTurmel)
* [ArthurHoaro](https://github.com/ArthurHoaro)
* [Astalaseven](https://github.com/Astalaseven)
* [Astyan-42](https://github.com/Astyan-42)
* [az5he6ch](https://github.com/az5he6ch)
* [azdkj532](https://github.com/azdkj532)
* [b1nj](https://github.com/b1nj)
* [benasse](https://github.com/benasse)
* [captn3m0](https://github.com/captn3m0)
* [chemel](https://github.com/chemel)
* [ckiw](https://github.com/ckiw)
* [cnlpete](https://github.com/cnlpete)
* [corenting](https://github.com/corenting)
* [couraudt](https://github.com/couraudt)
* [cyberjacob](https://github.com/cyberjacob)
* [da2x](https://github.com/da2x)
* [Daiyousei](https://github.com/Daiyousei)
* [dawidsowa](https://github.com/dawidsowa)
* [disk0x](https://github.com/disk0x)
* [DJCrashdummy](https://github.com/DJCrashdummy)
* [Djuuu](https://github.com/Djuuu)
* [DnAp](https://github.com/DnAp)
* [dominik-th](https://github.com/dominik-th)
* [Draeli](https://github.com/Draeli)
* [Dreckiger-Dan](https://github.com/Dreckiger-Dan)
* [em92](https://github.com/em92)
* [eMerzh](https://github.com/eMerzh)
* [EtienneM](https://github.com/EtienneM)
* [floviolleau](https://github.com/floviolleau)
* [fluffy-critter](https://github.com/fluffy-critter)
* [Frenzie](https://github.com/Frenzie)
* [fulmeek](https://github.com/fulmeek)
* [Ginko-Aloe](https://github.com/Ginko-Aloe)
* [Glandos](https://github.com/Glandos)
* [gloony](https://github.com/gloony)
* [GregThib](https://github.com/GregThib)
* [griffaurel](https://github.com/griffaurel)
* [Grummfy](https://github.com/Grummfy)
* [hunhejj](https://github.com/hunhejj)
* [husim0](https://github.com/husim0)
* [IceWreck](https://github.com/IceWreck)
* [j0k3r](https://github.com/j0k3r)
* [JackNUMBER](https://github.com/JackNUMBER)
* [jdigilio](https://github.com/jdigilio)
* [JeremyRand](https://github.com/JeremyRand)
* [Jocker666z](https://github.com/Jocker666z)
* [johnnygroovy](https://github.com/johnnygroovy)
* [killruana](https://github.com/killruana)
* [klimplant](https://github.com/klimplant)
* [kranack](https://github.com/kranack)
* [kraoc](https://github.com/kraoc)
* [l1n](https://github.com/l1n)
* [laBecasse](https://github.com/laBecasse)
* [lagaisse](https://github.com/lagaisse)
* [lalannev](https://github.com/lalannev)
* [ldidry](https://github.com/ldidry)
* [Leomaradan](https://github.com/Leomaradan)
* [Limero](https://github.com/Limero)
* [LogMANOriginal](https://github.com/LogMANOriginal)
* [lorenzos](https://github.com/lorenzos)
* [lukasklinger](https://github.com/lukasklinger)
* [m0zes](https://github.com/m0zes)
* [matthewseal](https://github.com/matthewseal)
* [mcbyte-it](https://github.com/mcbyte-it)
* [mdemoss](https://github.com/mdemoss)
* [melangue](https://github.com/melangue)
* [metaMMA](https://github.com/metaMMA)
* [mitsukarenai](https://github.com/mitsukarenai)
* [MonsieurPoutounours](https://github.com/MonsieurPoutounours)
* [mr-flibble](https://github.com/mr-flibble)
* [mro](https://github.com/mro)
* [mxmehl](https://github.com/mxmehl)
* [nel50n](https://github.com/nel50n)
* [niawag](https://github.com/niawag)
* [Nono-m0le](https://github.com/Nono-m0le)
* [ObsidianWitch](https://github.com/ObsidianWitch)
* [OliverParoczai](https://github.com/OliverParoczai)
* [oratosquilla-oratoria](https://github.com/oratosquilla-oratoria)
* [ORelio](https://github.com/ORelio)
* [PaulVayssiere](https://github.com/PaulVayssiere)
* [pellaeon](https://github.com/pellaeon)
* [Piranhaplant](https://github.com/Piranhaplant)
* [pit-fgfjiudghdf](https://github.com/pit-fgfjiudghdf)
* [pitchoule](https://github.com/pitchoule)
* [pmaziere](https://github.com/pmaziere)
* [Pofilo](https://github.com/Pofilo)
* [prysme01](https://github.com/prysme01)
* [quentinus95](https://github.com/quentinus95)
* [regisenguehard](https://github.com/regisenguehard)
* [Riduidel](https://github.com/Riduidel)
* [rogerdc](https://github.com/rogerdc)
* [Roliga](https://github.com/Roliga)
* [sebsauvage](https://github.com/sebsauvage)
* [shutosg](https://github.com/shutosg)
* [somini](https://github.com/somini)
* [squeek502](https://github.com/squeek502)
* [stjohnjohnson](https://github.com/stjohnjohnson)
* [Strubbl](https://github.com/Strubbl)
* [sublimz](https://github.com/sublimz)
* [sunchaserinfo](https://github.com/sunchaserinfo)
* [sysadminstory](https://github.com/sysadminstory)
* [tameroski](https://github.com/tameroski)
* [teromene](https://github.com/teromene)
* [thefranke](https://github.com/thefranke)
* [ThePadawan](https://github.com/ThePadawan)
* [TheRadialActive](https://github.com/TheRadialActive)
* [TitiTestScalingo](https://github.com/TitiTestScalingo)
* [triatic](https://github.com/triatic)
* [VerifiedJoseph](https://github.com/VerifiedJoseph)
* [WalterBarrett](https://github.com/WalterBarrett)
* [wtuuju](https://github.com/wtuuju)
* [xurxof](https://github.com/xurxof)
* [yardenac](https://github.com/yardenac)
* [ZeNairolf](https://github.com/ZeNairolf)
Licenses
===

View File

@ -0,0 +1,136 @@
<?php
/**
* This file is part of RSS-Bridge, a PHP project capable of generating RSS and
* Atom feeds for websites that don't have one.
*
* For the full license information, please view the UNLICENSE file distributed
* with this source code.
*
* @package Core
* @license http://unlicense.org/ UNLICENSE
* @link https://github.com/rss-bridge/rss-bridge
*/
/**
* Checks if the website for a given bridge is reachable.
*
* **Remarks**
* - This action is only available in debug mode.
* - Returns the bridge status as Json-formatted string.
* - Returns an error if the bridge is not whitelisted.
* - Returns a responsive web page that automatically checks all whitelisted
* bridges (using JavaScript) if no bridge is specified.
*/
class ConnectivityAction extends ActionAbstract {
public function execute() {
if(!Debug::isEnabled()) {
returnError('This action is only available in debug mode!');
}
if(!isset($this->userData['bridge'])) {
$this->returnEntryPage();
return;
}
$bridgeName = $this->userData['bridge'];
$this->reportBridgeConnectivity($bridgeName);
}
/**
* Generates a report about the bridge connectivity status and sends it back
* to the user.
*
* The report is generated as Json-formatted string in the format
* {
* "bridge": "<bridge-name>",
* "successful": true/false
* }
*
* @param string $bridgeName Name of the bridge to generate the report for
* @return void
*/
private function reportBridgeConnectivity($bridgeName) {
$bridgeFac = new \BridgeFactory();
$bridgeFac->setWorkingDir(PATH_LIB_BRIDGES);
if(!$bridgeFac->isWhitelisted($bridgeName)) {
header('Content-Type: text/html');
returnServerError('Bridge is not whitelisted!');
}
header('Content-Type: text/json');
$retVal = array(
'bridge' => $bridgeName,
'successful' => false,
'http_code' => 200,
);
$bridge = $bridgeFac->create($bridgeName);
if($bridge === false) {
echo json_encode($retVal);
return;
}
$curl_opts = array(
CURLOPT_CONNECTTIMEOUT => 5
);
try {
$reply = getContents($bridge::URI, array(), $curl_opts, true);
if($reply) {
$retVal['successful'] = true;
if (isset($reply['header'])) {
if (strpos($reply['header'], 'HTTP/1.1 301 Moved Permanently') !== false) {
$retVal['http_code'] = 301;
}
}
}
} catch(Exception $e) {
$retVal['successful'] = false;
}
echo json_encode($retVal);
}
private function returnEntryPage() {
echo <<<EOD
<!DOCTYPE html>
<html>
<head>
<link rel="stylesheet" href="static/bootstrap.min.css">
<link
rel="stylesheet"
href="https://use.fontawesome.com/releases/v5.6.3/css/all.css"
integrity="sha384-UHRtZLI+pbxtHCWp1t77Bi1L4ZtiqrqD80Kn4Z8NTSRyMA2Fd33n5dQ8lWUE00s/"
crossorigin="anonymous">
<link rel="stylesheet" href="static/connectivity.css">
<script src="static/connectivity.js" type="text/javascript"></script>
</head>
<body>
<div id="main-content" class="container">
<div class="progress">
<div class="progress-bar" role="progressbar" aria-valuenow="75" aria-valuemin="0" aria-valuemax="100"></div>
</div>
<div id="status-message" class="sticky-top alert alert-primary alert-dismissible fade show" role="alert">
<i id="status-icon" class="fas fa-sync"></i>
<span>...</span>
<button type="button" class="close" data-dismiss="alert" aria-label="Close" onclick="stopConnectivityChecks()">
<span aria-hidden="true">&times;</span>
</button>
</div>
<input type="text" class="form-control" id="search" onkeyup="search()" placeholder="Search for bridge..">
</div>
</body>
</html>
EOD;
}
}

View File

@ -19,13 +19,16 @@ class DetectAction extends ActionAbstract {
$format = $this->userData['format']
or returnClientError('You must specify a format!');
foreach(Bridge::getBridgeNames() as $bridgeName) {
$bridgeFac = new \BridgeFactory();
$bridgeFac->setWorkingDir(PATH_LIB_BRIDGES);
if(!Bridge::isWhitelisted($bridgeName)) {
foreach($bridgeFac->getBridgeNames() as $bridgeName) {
if(!$bridgeFac->isWhitelisted($bridgeName)) {
continue;
}
$bridge = Bridge::create($bridgeName);
$bridge = $bridgeFac->create($bridgeName);
if($bridge === false) {
continue;

View File

@ -12,26 +12,32 @@
*/
class DisplayAction extends ActionAbstract {
private function get_return_code($error) {
$returnCode = $error->getCode();
if ($returnCode === 301 || $returnCode === 302) {
# Don't pass redirect codes to the exterior
$returnCode = 508;
}
return $returnCode;
}
public function execute() {
$bridge = array_key_exists('bridge', $this->userData) ? $this->userData['bridge'] : null;
$format = $this->userData['format']
or returnClientError('You must specify a format!');
// DEPRECATED: 'nameFormat' scheme is replaced by 'name' in format parameter values
// this is to keep compatibility until futher complete removal
if(($pos = strpos($format, 'Format')) === (strlen($format) - strlen('Format'))) {
$format = substr($format, 0, $pos);
}
$bridgeFac = new \BridgeFactory();
$bridgeFac->setWorkingDir(PATH_LIB_BRIDGES);
// whitelist control
if(!Bridge::isWhitelisted($bridge)) {
if(!$bridgeFac->isWhitelisted($bridge)) {
throw new \Exception('This bridge is not whitelisted', 401);
die;
}
// Data retrieval
$bridge = Bridge::create($bridge);
$bridge = $bridgeFac->create($bridge);
$noproxy = array_key_exists('_noproxy', $this->userData)
&& filter_var($this->userData['_noproxy'], FILTER_VALIDATE_BOOLEAN);
@ -85,7 +91,9 @@ class DisplayAction extends ActionAbstract {
);
// Initialize cache
$cache = Cache::create(Configuration::getConfig('cache', 'type'));
$cacheFac = new CacheFactory();
$cacheFac->setWorkingDir(PATH_LIB_CACHES);
$cache = $cacheFac->create(Configuration::getConfig('cache', 'type'));
$cache->setScope('');
$cache->purgeCache(86400); // 24 hours
$cache->setKey($cache_params);
@ -147,63 +155,77 @@ class DisplayAction extends ActionAbstract {
} catch(Error $e) {
error_log($e);
$item = new \FeedItem();
if(logBridgeError($bridge::NAME, $e->getCode()) >= Configuration::getConfig('error', 'report_limit')) {
if(Configuration::getConfig('error', 'output') === 'feed') {
$item = new \FeedItem();
// Create "new" error message every 24 hours
$this->userData['_error_time'] = urlencode((int)(time() / 86400));
// Create "new" error message every 24 hours
$this->userData['_error_time'] = urlencode((int)(time() / 86400));
// Error 0 is a special case (i.e. "trying to get property of non-object")
if($e->getCode() === 0) {
$item->setTitle(
'Bridge encountered an unexpected situation! ('
. $this->userData['_error_time']
. ')'
);
} else {
$item->setTitle(
'Bridge returned error '
. $e->getCode()
. '! ('
. $this->userData['_error_time']
. ')'
);
// Error 0 is a special case (i.e. "trying to get property of non-object")
if($e->getCode() === 0) {
$item->setTitle(
'Bridge encountered an unexpected situation! ('
. $this->userData['_error_time']
. ')'
);
} else {
$item->setTitle(
'Bridge returned error '
. $e->getCode()
. '! ('
. $this->userData['_error_time']
. ')'
);
}
$item->setURI(
(isset($_SERVER['REQUEST_URI']) ? parse_url($_SERVER['REQUEST_URI'], PHP_URL_PATH) : '')
. '?'
. http_build_query($this->userData)
);
$item->setTimestamp(time());
$item->setContent(buildBridgeException($e, $bridge));
$items[] = $item;
} elseif(Configuration::getConfig('error', 'output') === 'http') {
header('Content-Type: text/html', true, $this->get_return_code($e));
die(buildTransformException($e, $bridge));
}
}
$item->setURI(
(isset($_SERVER['REQUEST_URI']) ? parse_url($_SERVER['REQUEST_URI'], PHP_URL_PATH) : '')
. '?'
. http_build_query($this->userData)
);
$item->setTimestamp(time());
$item->setContent(buildBridgeException($e, $bridge));
$items[] = $item;
} catch(Exception $e) {
error_log($e);
$item = new \FeedItem();
if(logBridgeError($bridge::NAME, $e->getCode()) >= Configuration::getConfig('error', 'report_limit')) {
if(Configuration::getConfig('error', 'output') === 'feed') {
$item = new \FeedItem();
// Create "new" error message every 24 hours
$this->userData['_error_time'] = urlencode((int)(time() / 86400));
// Create "new" error message every 24 hours
$this->userData['_error_time'] = urlencode((int)(time() / 86400));
$item->setURI(
(isset($_SERVER['REQUEST_URI']) ? parse_url($_SERVER['REQUEST_URI'], PHP_URL_PATH) : '')
. '?'
. http_build_query($this->userData)
);
$item->setURI(
(isset($_SERVER['REQUEST_URI']) ? parse_url($_SERVER['REQUEST_URI'], PHP_URL_PATH) : '')
. '?'
. http_build_query($this->userData)
);
$item->setTitle(
'Bridge returned error '
. $e->getCode()
. '! ('
. $this->userData['_error_time']
. ')'
);
$item->setTimestamp(time());
$item->setContent(buildBridgeException($e, $bridge));
$item->setTitle(
'Bridge returned error '
. $e->getCode()
. '! ('
. $this->userData['_error_time']
. ')'
);
$item->setTimestamp(time());
$item->setContent(buildBridgeException($e, $bridge));
$items[] = $item;
$items[] = $item;
} elseif(Configuration::getConfig('error', 'output') === 'http') {
header('Content-Type: text/html', true, $this->get_return_code($e));
die(buildTransformException($e, $bridge));
}
}
}
// Store data in cache
@ -216,7 +238,9 @@ class DisplayAction extends ActionAbstract {
// Data transformation
try {
$format = Format::create($format);
$formatFac = new FormatFactory();
$formatFac->setWorkingDir(PATH_LIB_FORMATS);
$format = $formatFac->create($format);
$format->setItems($items);
$format->setExtraInfos($infos);
$format->setLastModified($cache->getTime());

View File

@ -17,9 +17,12 @@ class ListAction extends ActionAbstract {
$list->bridges = array();
$list->total = 0;
foreach(Bridge::getBridgeNames() as $bridgeName) {
$bridgeFac = new \BridgeFactory();
$bridgeFac->setWorkingDir(PATH_LIB_BRIDGES);
$bridge = Bridge::create($bridgeName);
foreach($bridgeFac->getBridgeNames() as $bridgeName) {
$bridge = $bridgeFac->create($bridgeName);
if($bridge === false) { // Broken bridge, show as inactive
@ -31,7 +34,7 @@ class ListAction extends ActionAbstract {
}
$status = Bridge::isWhitelisted($bridgeName) ? 'active' : 'inactive';
$status = $bridgeFac->isWhitelisted($bridgeName) ? 'active' : 'inactive';
$list->bridges[$bridgeName] = array(
'status' => $status,

View File

@ -134,11 +134,11 @@ EOT;
// data-asin="B00WTHJ5SU" data-asin-price="14.99" data-asin-shipping="0"
// data-asin-currency-code="USD" data-substitute-count="-1" ... />
if ($asinData) {
return [
return array(
'price' => $asinData->getAttribute('data-asin-price'),
'currency' => $asinData->getAttribute('data-asin-currency-code'),
'shipping' => $asinData->getAttribute('data-asin-shipping')
];
);
}
return false;
@ -150,11 +150,11 @@ EOT;
preg_match('/^\s*([A-Z]{3}|£|\$)\s?([\d.,]+)\s*$/', $priceDiv->plaintext, $matches);
if (count($matches) === 3) {
return [
return array(
'price' => $matches[2],
'currency' => $matches[1],
'shipping' => '0'
];
);
}
return false;

View File

@ -0,0 +1,149 @@
<?php
class AppleAppStoreBridge extends BridgeAbstract {
const MAINTAINER = 'captn3m0';
const NAME = 'Apple App Store';
const URI = 'https://apps.apple.com/';
const CACHE_TIMEOUT = 3600; // 1h
const DESCRIPTION = 'Returns version updates for a specific application';
const PARAMETERS = array(array(
'id' => array(
'name' => 'Application ID',
'required' => true,
'exampleValue' => '310633997'
),
'p' => array(
'name' => 'Platform',
'type' => 'list',
'values' => array(
'iPad' => 'ipad',
'iPhone' => 'iphone',
'Mac' => 'mac',
// The following 2 are present in responses
// but not yet tested
'Web' => 'web',
'Apple TV' => 'appletv',
),
'defaultValue' => 'iphone',
),
'country' => array(
'name' => 'Store Country',
'type' => 'list',
'values' => array(
'US' => 'US',
'India' => 'IN',
'Canada' => 'CA'
),
'defaultValue' => 'US',
),
));
const PLATFORM_MAPPING = array(
'iphone' => 'ios',
'ipad' => 'ios',
);
private function makeHtmlUrl($id, $country){
return 'https://apps.apple.com/' . $country . '/app/id' . $id;
}
private function makeJsonUrl($id, $platform, $country){
return "https://amp-api.apps.apple.com/v1/catalog/$country/apps/$id?platform=$platform&extend=versionHistory";
}
public function getName(){
if (isset($this->name)) {
return $this->name . ' - AppStore Updates';
}
return parent::getName();
}
/**
* In case of some platforms, the data is present in the initial response
*/
private function getDataFromShoebox($id, $platform, $country){
$uri = $this->makeHtmlUrl($id, $country);
$html = getSimpleHTMLDOMCached($uri, 3600);
$script = $html->find('script[id="shoebox-ember-data-store"]', 0);
$json = json_decode($script->innertext, true);
return $json['data'];
}
private function getJWTToken($id, $platform, $country){
$uri = $this->makeHtmlUrl($id, $country);
$html = getSimpleHTMLDOMCached($uri, 3600);
$meta = $html->find('meta[name="web-experience-app/config/environment"]', 0);
$json = urldecode($meta->content);
$json = json_decode($json);
return $json->MEDIA_API->token;
}
private function getAppData($id, $platform, $country, $token){
$uri = $this->makeJsonUrl($id, $platform, $country);
$headers = array(
"Authorization: Bearer $token",
);
$json = json_decode(getContents($uri, $headers), true);
return $json['data'][0];
}
/**
* Parses the version history from the data received
* @return array list of versions with details on each element
*/
private function getVersionHistory($data, $platform){
switch($platform) {
case 'mac':
return $data['relationships']['platforms']['data'][0]['attributes']['versionHistory'];
default:
$os = self::PLATFORM_MAPPING[$platform];
return $data['attributes']['platformAttributes'][$os]['versionHistory'];
}
}
public function collectData() {
$id = $this->getInput('id');
$country = $this->getInput('country');
$platform = $this->getInput('p');
switch ($platform) {
case 'mac':
$data = $this->getDataFromShoebox($id, $platform, $country);
break;
default:
$token = $this->getJWTToken($id, $platform, $country);
$data = $this->getAppData($id, $platform, $country, $token);
}
$versionHistory = $this->getVersionHistory($data, $platform);
$name = $this->name = $data['attributes']['name'];
$author = $data['attributes']['artistName'];
foreach ($versionHistory as $row) {
$item = array();
$item['content'] = nl2br($row['releaseNotes']);
$item['title'] = $name . ' - ' . $row['versionDisplay'];
$item['timestamp'] = $row['releaseDate'];
$item['author'] = $author;
$item['uri'] = $this->makeHtmlUrl($id, $country);
$this->items[] = $item;
}
}
}

View File

@ -5,19 +5,19 @@ class AppleMusicBridge extends BridgeAbstract {
const URI = 'https://www.apple.com';
const DESCRIPTION = 'Fetches the latest releases from an artist';
const MAINTAINER = 'Limero';
const PARAMETERS = [[
'url' => [
const PARAMETERS = array(array(
'url' => array(
'name' => 'Artist URL',
'exampleValue' => 'https://itunes.apple.com/us/artist/dunderpatrullen/329796274',
'required' => true,
],
'imgSize' => [
),
'imgSize' => array(
'name' => 'Image size for thumbnails (in px)',
'type' => 'number',
'defaultValue' => 512,
'required' => true,
]
]];
)
));
const CACHE_TIMEOUT = 21600; // 6 hours
public function collectData() {
@ -36,12 +36,12 @@ class AppleMusicBridge extends BridgeAbstract {
// Loop through each object
foreach ($json->included as $obj) {
if ($obj->type === 'lockup/album') {
$this->items[] = [
$this->items[] = array(
'title' => $obj->attributes->artistName . ' - ' . $obj->attributes->name,
'uri' => $obj->attributes->url,
'timestamp' => $obj->attributes->releaseDate,
'enclosures' => $obj->relationships->artwork->data->id,
];
);
} elseif ($obj->type === 'image') {
$images[$obj->id] = $obj->attributes->url;
}
@ -49,9 +49,9 @@ class AppleMusicBridge extends BridgeAbstract {
// Add the images to each item
foreach ($this->items as &$item) {
$item['enclosures'] = [
$item['enclosures'] = array(
str_replace('{w}x{h}bb.{f}', $imgSize . 'x0w.jpg', $images[$item['enclosures']]),
];
);
}
// Sort the order to put the latest albums first

File diff suppressed because it is too large Load Diff

View File

@ -15,16 +15,6 @@ class AutoJMBridge extends BridgeAbstract {
'title' => 'URL d\'une recherche avec filtre de véhicules sans le http://www.autojm.fr/',
'exampleValue' => 'achat-voitures-neuves-peugeot-nouvelle-308-5p'
),
'isDispo' => array(
'name' => 'Disponibilité',
'type' => 'list',
'values' => array(
'-' => '',
'En stock' => 1,
'Sur commande' => 0
),
'title' => 'Critère de disponibilité'
),
'energy' => array(
'name' => 'Carburant',
'type' => 'list',
@ -92,7 +82,6 @@ class AutoJMBridge extends BridgeAbstract {
// Build the form
$post_data = array(
'form[isDispo]' => $this->getInput('isDispo'),
'form[energy]' => $this->getInput('energy'),
'form[transmission]' => $this->getInput('transmission'),
'form[priceMin]' => $this->getInput('priceMin'),
@ -121,7 +110,7 @@ class AutoJMBridge extends BridgeAbstract {
$html = str_get_html($data->content);
// Go through every finisha of the model
$list = $html->find('h2');
$list = $html->find('h3');
foreach ($list as $finish) {
$finish_name = $finish->plaintext;
$motorizations = $finish->next_sibling()->find('li');

View File

@ -55,9 +55,7 @@ class BAEBridge extends BridgeAbstract {
$content .= '<hr>';
$content .= $htmlDetail->find('section', 0)->innertext;
$content = str_replace('src="/', 'src="' . parent::getURI() . '/', $content);
$content = str_replace('href="/', 'href="' . parent::getURI() . '/', $content);
$item['content'] = $content;
$item['content'] = defaultLinkTo($content, parent::getURI());
$image = $htmlDetail->find('#zoom', 0);
if ($image) {
$item['enclosures'] = array(parent::getURI() . $image->getAttribute('src'));

View File

@ -1,73 +1,262 @@
<?php
class BandcampBridge extends BridgeAbstract {
const MAINTAINER = 'sebsauvage';
const NAME = 'Bandcamp Tag';
const MAINTAINER = 'sebsauvage, Roliga';
const NAME = 'Bandcamp Bridge';
const URI = 'https://bandcamp.com/';
const CACHE_TIMEOUT = 600; // 10min
const DESCRIPTION = 'New bandcamp release by tag';
const PARAMETERS = array( array(
'tag' => array(
'name' => 'tag',
'type' => 'text',
'required' => true
const DESCRIPTION = 'New bandcamp releases by tag, band or album';
const PARAMETERS = array(
'By tag' => array(
'tag' => array(
'name' => 'tag',
'type' => 'text',
'required' => true
)
),
'By band' => array(
'band' => array(
'name' => 'band',
'type' => 'text',
'title' => 'Band name as seen in the band page URL',
'required' => true
),
'type' => array(
'name' => 'Articles are',
'type' => 'list',
'values' => array(
'Releases' => 'releases',
'Releases, new one when track list changes' => 'changes',
'Individual tracks' => 'tracks'
),
'defaultValue' => 'changes'
),
'limit' => array(
'name' => 'limit',
'type' => 'number',
'title' => 'Number of releases to return',
'defaultValue' => 5
)
),
'By album' => array(
'band' => array(
'name' => 'band',
'type' => 'text',
'title' => 'Band name as seen in the album page URL',
'required' => true
),
'album' => array(
'name' => 'album',
'type' => 'text',
'title' => 'Album name as seen in the album page URL',
'required' => true
),
'type' => array(
'name' => 'Articles are',
'type' => 'list',
'values' => array(
'Releases' => 'releases',
'Releases, new one when track list changes' => 'changes',
'Individual tracks' => 'tracks'
),
'defaultValue' => 'tracks'
)
)
));
);
const IMGURI = 'https://f4.bcbits.com/';
const IMGSIZE_300PX = 23;
const IMGSIZE_700PX = 16;
private $feedName;
public function getIcon() {
return 'https://s4.bcbits.com/img/bc_favicon.ico';
}
public function collectData(){
$url = self::URI . 'api/hub/1/dig_deeper';
$data = $this->buildRequestJson();
$header = array(
'Content-Type: application/json',
'Content-Length: ' . strlen($data)
);
$opts = array(
CURLOPT_CUSTOMREQUEST => 'POST',
CURLOPT_POSTFIELDS => $data
);
$content = getContents($url, $header, $opts)
or returnServerError('Could not complete request to: ' . $url);
$json = json_decode($content);
if ($json->ok !== true) {
returnServerError('Invalid response');
}
foreach ($json->items as $entry) {
$url = $entry->tralbum_url;
$artist = $entry->artist;
$title = $entry->title;
// e.g. record label is the releaser, but not the artist
$releaser = $entry->band_name !== $entry->artist ? $entry->band_name : null;
$full_title = $artist . ' - ' . $title;
$full_artist = $artist;
if (isset($releaser)) {
$full_title .= ' (' . $releaser . ')';
$full_artist .= ' (' . $releaser . ')';
}
$small_img = $this->getImageUrl($entry->art_id, self::IMGSIZE_300PX);
$img = $this->getImageUrl($entry->art_id, self::IMGSIZE_700PX);
$item = array(
'uri' => $url,
'author' => $full_artist,
'title' => $full_title
switch($this->queriedContext) {
case 'By tag':
$url = self::URI . 'api/hub/1/dig_deeper';
$data = $this->buildRequestJson();
$header = array(
'Content-Type: application/json',
'Content-Length: ' . strlen($data)
);
$item['content'] = "<img src='$small_img' /><br/>$full_title";
$item['enclosures'] = array($img);
$this->items[] = $item;
$opts = array(
CURLOPT_CUSTOMREQUEST => 'POST',
CURLOPT_POSTFIELDS => $data
);
$content = getContents($url, $header, $opts)
or returnServerError('Could not complete request to: ' . $url);
$json = json_decode($content);
if ($json->ok !== true) {
returnServerError('Invalid response');
}
foreach ($json->items as $entry) {
$url = $entry->tralbum_url;
$artist = $entry->artist;
$title = $entry->title;
// e.g. record label is the releaser, but not the artist
$releaser = $entry->band_name !== $entry->artist ? $entry->band_name : null;
$full_title = $artist . ' - ' . $title;
$full_artist = $artist;
if (isset($releaser)) {
$full_title .= ' (' . $releaser . ')';
$full_artist .= ' (' . $releaser . ')';
}
$small_img = $this->getImageUrl($entry->art_id, self::IMGSIZE_300PX);
$img = $this->getImageUrl($entry->art_id, self::IMGSIZE_700PX);
$item = array(
'uri' => $url,
'author' => $full_artist,
'title' => $full_title
);
$item['content'] = "<img src='$small_img' /><br/>$full_title";
$item['enclosures'] = array($img);
$this->items[] = $item;
}
break;
case 'By band':
case 'By album':
$html = getSimpleHTMLDOMCached($this->getURI(), 86400);
$titleElement = $html->find('head meta[name=title]', 0)
or returnServerError('Unable to find title on: ' . $this->getURI());
$this->feedName = $titleElement->content;
$regex = '/band_id=(\d+)/';
if(preg_match($regex, $html, $matches) == false)
returnServerError('Unable to find band ID on: ' . $this->getURI());
$band_id = $matches[1];
$tralbums = array();
switch($this->queriedContext) {
case 'By band':
$query_data = array(
'band_id' => $band_id
);
$band_data = $this->apiGet('mobile/22/band_details', $query_data);
$num_albums = min(count($band_data->discography), $this->getInput('limit'));
for($i = 0; $i < $num_albums; $i++) {
$album_basic_data = $band_data->discography[$i];
// 'a' or 't' for albums and individual tracks respectively
$tralbum_type = substr($album_basic_data->item_type, 0, 1);
$query_data = array(
'band_id' => $band_id,
'tralbum_type' => $tralbum_type,
'tralbum_id' => $album_basic_data->item_id
);
$tralbums[] = $this->apiGet('mobile/22/tralbum_details', $query_data);
}
break;
case 'By album':
$regex = '/album=(\d+)/';
if(preg_match($regex, $html, $matches) == false)
returnServerError('Unable to find album ID on: ' . $this->getURI());
$album_id = $matches[1];
$query_data = array(
'band_id' => $band_id,
'tralbum_type' => 'a',
'tralbum_id' => $album_id
);
$tralbums[] = $this->apiGet('mobile/22/tralbum_details', $query_data);
break;
}
foreach ($tralbums as $tralbum_data) {
if ($tralbum_data->type === 'a' && $this->getInput('type') === 'tracks') {
foreach ($tralbum_data->tracks as $track) {
$query_data = array(
'band_id' => $band_id,
'tralbum_type' => 't',
'tralbum_id' => $track->track_id
);
$track_data = $this->apiGet('mobile/22/tralbum_details', $query_data);
$this->items[] = $this->buildTralbumItem($track_data);
}
} else {
$this->items[] = $this->buildTralbumItem($tralbum_data);
}
}
break;
}
}
private function buildTralbumItem($tralbum_data){
$band_data = $tralbum_data->band;
// Format title like: ARTIST - ALBUM/TRACK (OPTIONAL RELEASER)
// Format artist/author like: ARTIST (OPTIONAL RELEASER)
//
// If the album/track is released under a label/a band other than the artist
// themselves, append that releaser name to the title and artist/author.
//
// This sadly doesn't always work right for individual tracks as the artist
// of the track is always set to the releaser.
$artist = $tralbum_data->tralbum_artist;
$full_title = $artist . ' - ' . $tralbum_data->title;
$full_artist = $artist;
if (isset($tralbum_data->label)) {
$full_title .= ' (' . $tralbum_data->label . ')';
$full_artist .= ' (' . $tralbum_data->label . ')';
} elseif ($band_data->name !== $artist) {
$full_title .= ' (' . $band_data->name . ')';
$full_artist .= ' (' . $band_data->name . ')';
}
$small_img = $this->getImageUrl($tralbum_data->art_id, self::IMGSIZE_300PX);
$img = $this->getImageUrl($tralbum_data->art_id, self::IMGSIZE_700PX);
$item = array(
'uri' => $tralbum_data->bandcamp_url,
'author' => $full_artist,
'title' => $full_title,
'enclosures' => array($img),
'timestamp' => $tralbum_data->release_date
);
$item['categories'] = array();
foreach ($tralbum_data->tags as $tag) {
$item['categories'][] = $tag->norm_name;
}
// Give articles a unique UID depending on its track list
// Releases should then show up as new articles when tracks are added
if ($this->getInput('type') === 'changes') {
$item['uid'] = "bandcamp/$band_data->band_id/$tralbum_data->id/";
foreach ($tralbum_data->tracks as $track) {
$item['uid'] .= $track->track_id;
}
}
$item['content'] = "<img src='$small_img' /><br/>$full_title<br/>";
if ($tralbum_data->type === 'a') {
$item['content'] .= '<ol>';
foreach ($tralbum_data->tracks as $track) {
$item['content'] .= "<li>$track->title</li>";
}
$item['content'] .= '</ol>';
}
if (!empty($tralbum_data->about)) {
$item['content'] .= '<p>'
. nl2br($tralbum_data->about)
. '</p>';
}
return $item;
}
private function buildRequestJson(){
$requestJson = array(
'tag' => $this->getInput('tag'),
@ -81,11 +270,94 @@ class BandcampBridge extends BridgeAbstract {
return self::IMGURI . 'img/a' . $id . '_' . $size . '.jpg';
}
private function apiGet($endpoint, $query_data) {
$url = self::URI . 'api/' . $endpoint . '?' . http_build_query($query_data);
$data = json_decode(getContents($url))
or returnServerError('API request to "' . $url . '" failed.');
return $data;
}
public function getURI(){
switch($this->queriedContext) {
case 'By tag':
if(!is_null($this->getInput('tag'))) {
return self::URI
. 'tag/'
. urlencode($this->getInput('tag'))
. '?sort_field=date';
}
break;
case 'By band':
if(!is_null($this->getInput('band'))) {
return 'https://'
. $this->getInput('band')
. '.bandcamp.com/music';
}
break;
case 'By album':
if(!is_null($this->getInput('band')) && !is_null($this->getInput('album'))) {
return 'https://'
. $this->getInput('band')
. '.bandcamp.com/album/'
. $this->getInput('album');
}
break;
}
return parent::getURI();
}
public function getName(){
if(!is_null($this->getInput('tag'))) {
return $this->getInput('tag') . ' - Bandcamp Tag';
switch($this->queriedContext) {
case 'By tag':
if(!is_null($this->getInput('tag'))) {
return $this->getInput('tag') . ' - Bandcamp Tag';
}
break;
case 'By band':
if(isset($this->feedName)) {
return $this->feedName . ' - Bandcamp Band';
} elseif(!is_null($this->getInput('band'))) {
return $this->getInput('band') . ' - Bandcamp Band';
}
break;
case 'By album':
if(isset($this->feedName)) {
return $this->feedName . ' - Bandcamp Album';
} elseif(!is_null($this->getInput('album'))) {
return $this->getInput('album') . ' - Bandcamp Album';
}
break;
}
return parent::getName();
}
public function detectParameters($url) {
$params = array();
// By tag
$regex = '/^(https?:\/\/)?bandcamp\.com\/tag\/([^\/.&?\n]+)/';
if(preg_match($regex, $url, $matches) > 0) {
$params['tag'] = urldecode($matches[2]);
return $params;
}
// By band
$regex = '/^(https?:\/\/)?([^\/.&?\n]+?)\.bandcamp\.com/';
if(preg_match($regex, $url, $matches) > 0) {
$params['band'] = urldecode($matches[2]);
return $params;
}
// By album
$regex = '/^(https?:\/\/)?([^\/.&?\n]+?)\.bandcamp\.com\/album\/([^\/.&?\n]+)/';
if(preg_match($regex, $url, $matches) > 0) {
$params['band'] = urldecode($matches[2]);
$params['album'] = urldecode($matches[3]);
return $params;
}
return null;
}
}

View File

@ -3,17 +3,11 @@ class BastaBridge extends BridgeAbstract {
const MAINTAINER = 'qwertygc';
const NAME = 'Bastamag Bridge';
const URI = 'http://www.bastamag.net/';
const URI = 'https://www.bastamag.net/';
const CACHE_TIMEOUT = 7200; // 2h
const DESCRIPTION = 'Returns the newest articles.';
public function collectData(){
// Replaces all relative image URLs by absolute URLs.
// Relative URLs always start with 'local/'!
function replaceImageUrl($content){
return preg_replace('/src=["\']{1}([^"\']+)/ims', 'src=\'' . self::URI . '$1\'', $content);
}
$html = getSimpleHTMLDOM(self::URI . 'spip.php?page=backend')
or returnServerError('Could not request Bastamag.');
@ -25,7 +19,13 @@ class BastaBridge extends BridgeAbstract {
$item['title'] = $element->find('title', 0)->innertext;
$item['uri'] = $element->find('guid', 0)->plaintext;
$item['timestamp'] = strtotime($element->find('dc:date', 0)->plaintext);
$item['content'] = replaceImageUrl(getSimpleHTMLDOM($item['uri'])->find('div.texte', 0)->innertext);
// Replaces all relative image URLs by absolute URLs.
// Relative URLs always start with 'local/'!
$item['content'] = preg_replace(
'/src=["\']{1}([^"\']+)/ims',
'src=\'' . self::URI . '$1\'',
getSimpleHTMLDOM($item['uri'])->find('div.texte', 0)->innertext
);
$this->items[] = $item;
$limit++;
}

View File

@ -92,7 +92,7 @@ class BingSearchBridge extends BridgeAbstract
or returnServerError('Could not request ' . self::NAME);
$sizeKey = $this->getInput('image_size');
$items = [];
$items = array();
foreach ($html->find('a.iusc') as $element) {
$data = json_decode(htmlspecialchars_decode($element->getAttribute('m')), true);

View File

@ -1,69 +0,0 @@
<?php
class BloombergBridge extends BridgeAbstract
{
const NAME = 'Bloomberg';
const URI = 'https://www.bloomberg.com/';
const DESCRIPTION = 'Trending stories from Bloomberg';
const MAINTAINER = 'mdemoss';
const PARAMETERS = array(
'Trending Stories' => array(),
'From Search' => array(
'q' => array(
'name' => 'Keyword',
'required' => true
)
)
);
public function getName()
{
switch($this->queriedContext) {
case 'Trending Stories':
return self::NAME . ' Trending Stories';
case 'From Search':
if (!is_null($this->getInput('q'))) {
return self::NAME . ' Search : ' . $this->getInput('q');
}
break;
}
return parent::getName();
}
public function getIcon() {
return 'https://assets.bwbx.io/s3/javelin/public/hub/images/favicon-black-63fe5249d3.png';
}
public function collectData()
{
switch($this->queriedContext) {
case 'Trending Stories': // Get list of top new <article>s from the front page.
$html = getSimpleHTMLDOMCached($this->getURI(), 300);
$stories = $html->find('ul.top-news-v3__stories article.top-news-v3-story');
break;
case 'From Search': // Get list of <article> elements from search.
$html = getSimpleHTMLDOMCached(
$this->getURI() .
'search?sort=time:desc&page=1&query=' .
urlencode($this->getInput('q')), 300
);
$stories = $html->find('div.search-result-items article.search-result-story');
break;
}
foreach ($stories as $element) {
$item['uri'] = $element->find('h1 a', 0)->href;
if (preg_match('#^https://#i', $item['uri']) !== 1) {
$item['uri'] = $this->getURI() . $item['uri'];
}
$articleHtml = getSimpleHTMLDOMCached($item['uri']);
if (!$articleHtml) {
continue;
}
$item['title'] = $element->find('h1 a', 0)->plaintext;
$item['timestamp'] = strtotime($articleHtml->find('meta[name=iso-8601-publish-date],meta[name=date]', 0)->content);
$item['content'] = $articleHtml->find('meta[name=description]', 0)->content;
$this->items[] = $item;
}
}
}

View File

@ -92,6 +92,21 @@ class BrutBridge extends BridgeAbstract {
return parent::getURI();
}
public function getName() {
if (!is_null($this->getInput('edition')) && !is_null($this->getInput('category'))) {
$parameters = $this->getParameters();
$editionValues = array_flip($parameters[0]['edition']['values']);
$categoryValues = array_flip($parameters[0]['category']['values']);
return $categoryValues[$this->getInput('category')] . ' - ' .
$editionValues[$this->getInput('edition')] . ' - Brut.';
}
return parent::getName();
}
private function processDate($description) {
if ($this->getInput('edition') === 'uk') {

View File

@ -0,0 +1,63 @@
<?php
class CNETFranceBridge extends FeedExpander
{
const MAINTAINER = 'leomaradan';
const NAME = 'CNET France';
const URI = 'https://www.cnetfrance.fr/';
const CACHE_TIMEOUT = 3600; // 1h
const DESCRIPTION = 'CNET France RSS with filters';
const PARAMETERS = array(
'filters' => array(
'title' => array(
'name' => 'Exclude by title',
'required' => false,
'title' => 'Title term, separated by semicolon (;)',
'defaultValue' => 'bon plan;bons plans;au meilleur prix;des meilleures offres;Amazon Prime Day;RED by SFR ou B&You'
),
'url' => array(
'name' => 'Exclude by url',
'required' => false,
'title' => 'URL term, separated by semicolon (;)',
'defaultValue' => 'bon-plan;bons-plans'
)
)
);
private $bannedTitle = array();
private $bannedURL = array();
public function collectData()
{
$title = $this->getInput('title');
$url = $this->getInput('url');
if ($title !== null) {
$this->bannedTitle = explode(';', $title);
}
if ($url !== null) {
$this->bannedURL = explode(';', $url);
}
$this->collectExpandableDatas('https://www.cnetfrance.fr/feeds/rss/news/');
}
protected function parseItem($feedItem)
{
$item = parent::parseItem($feedItem);
foreach ($this->bannedTitle as $term) {
if (preg_match('/' . $term . '/mi', $item['title']) === 1) {
return null;
}
}
foreach ($this->bannedURL as $term) {
if (preg_match('/' . $term . '/mi', $item['uri']) === 1) {
return null;
}
}
return $item;
}
}

View File

@ -22,7 +22,7 @@ class CachetBridge extends BridgeAbstract {
);
const CACHE_TIMEOUT = 300;
private $componentCache = [];
private $componentCache = array();
public function getURI() {
return $this->getInput('host') === null ? 'https://cachethq.io/' : $this->getInput('host');
@ -114,13 +114,13 @@ class CachetBridge extends BridgeAbstract {
$uidOrig = $permalink . $incident->created_at;
$uid = hash('sha512', $uidOrig);
$timestamp = strtotime($incident->created_at);
$categories = [];
$categories = array();
$categories[] = $incident->human_status;
if ($componentName !== '') {
$categories[] = $componentName;
}
$item = [];
$item = array();
$item['uri'] = $permalink;
$item['title'] = $title;
$item['timestamp'] = $timestamp;

View File

@ -2,7 +2,7 @@
class CastorusBridge extends BridgeAbstract {
const MAINTAINER = 'logmanoriginal';
const NAME = 'Castorus Bridge';
const URI = 'http://www.castorus.com';
const URI = 'https://www.castorus.com';
const CACHE_TIMEOUT = 600; // 10min
const DESCRIPTION = 'Returns the latest changes';
@ -83,7 +83,7 @@ class CastorusBridge extends BridgeAbstract {
if(!$html)
returnServerError('Could not load data from ' . self::URI . '!');
$activities = $html->find('div#activite/li');
$activities = $html->find('div#activite > li');
if(!$activities)
returnServerError('Failed to find activities!');

View File

@ -3,7 +3,7 @@ class CollegeDeFranceBridge extends BridgeAbstract {
const MAINTAINER = 'pit-fgfjiudghdf';
const NAME = 'CollegeDeFrance';
const URI = 'http://www.college-de-france.fr/';
const URI = 'https://www.college-de-france.fr/';
const CACHE_TIMEOUT = 10800; // 3h
const DESCRIPTION = 'Returns the latest audio and video from CollegeDeFrance';

View File

@ -0,0 +1,65 @@
<?php
class ComicsKingdomBridge extends BridgeAbstract {
const MAINTAINER = 'stjohnjohnson';
const NAME = 'Comics Kingdom Unofficial RSS';
const URI = 'https://www.comicskingdom.com/';
const CACHE_TIMEOUT = 21600; // 6h
const DESCRIPTION = 'Comics Kingdom Unofficial RSS';
const PARAMETERS = array( array(
'comicname' => array(
'name' => 'comicname',
'type' => 'text',
'required' => true
)
));
public function collectData(){
$html = getSimpleHTMLDOM($this->getURI(), array(), array(), true, false)
or returnServerError('Could not request Comics Kingdom: ' . $this->getURI());
// Get author from first page
$author = $html->find('div.author p', 0)->plaintext
or returnServerError('Comics Kingdom comic does not exist: ' . $this->getURI());;
// Get current date/link
$link = $html->find('meta[property=og:url]', 0)->content;
for($i = 0; $i < 5; $i++) {
$item = array();
$page = getSimpleHTMLDOM($link)
or returnServerError('Could not request Comics Kingdom: ' . $link);
$imagelink = $page->find('meta[property=og:image]', 0)->content;
$prevSlug = $page->find('slider-arrow[:is-left-arrow=true]', 0);
$link = $this->getURI() . '/' . $prevSlug->getAttribute('date-slug');
$date = explode('/', $link);
$item['id'] = $imagelink;
$item['uri'] = $link;
$item['author'] = $author;
$item['title'] = 'Comics Kingdom ' . $this->getInput('comicname');
$item['timestamp'] = DateTime::createFromFormat('Y-m-d', $date[count($date) - 1])->getTimestamp();
$item['content'] = '<img src="' . $imagelink . '" />';
$this->items[] = $item;
}
}
public function getURI(){
if(!is_null($this->getInput('comicname'))) {
return self::URI . urlencode($this->getInput('comicname'));
}
return parent::getURI();
}
public function getName(){
if(!is_null($this->getInput('comicname'))) {
return $this->getInput('comicname') . ' - Comics Kingdom';
}
return parent::getName();
}
}

View File

@ -10,20 +10,20 @@ class ContainerLinuxReleasesBridge extends BridgeAbstract {
const BETA = 'beta';
const ALPHA = 'alpha';
const PARAMETERS = [
[
'channel' => [
const PARAMETERS = array(
array(
'channel' => array(
'name' => 'Release Channel',
'type' => 'list',
'defaultValue' => self::STABLE,
'values' => [
'values' => array(
'Stable' => self::STABLE,
'Beta' => self::BETA,
'Alpha' => self::ALPHA,
],
]
]
];
),
)
)
);
private function getReleaseFeed($jsonUrl) {
$json = getContents($jsonUrl)
@ -39,7 +39,7 @@ class ContainerLinuxReleasesBridge extends BridgeAbstract {
$data = $this->getReleaseFeed($this->getJsonUri());
foreach ($data as $releaseVersion => $release) {
$item = [];
$item = array();
$item['uri'] = "https://coreos.com/releases/#$releaseVersion";
$item['title'] = $releaseVersion;

View File

@ -0,0 +1,109 @@
<?php
class CuriousCatBridge extends BridgeAbstract {
const NAME = 'Curious Cat Bridge';
const URI = 'https://curiouscat.me';
const DESCRIPTION = 'Returns list of newest questions and answers for a user profile';
const MAINTAINER = 'VerifiedJoseph';
const PARAMETERS = array(array(
'username' => array(
'name' => 'Username',
'type' => 'text',
'required' => true,
'exampleValue' => 'koethekoethe',
)
));
const CACHE_TIMEOUT = 3600;
public function collectData() {
$url = self::URI . '/api/v2/profile?username=' . urlencode($this->getInput('username'));
$apiJson = getContents($url)
or returnServerError('Could not request: ' . $url);
$apiData = json_decode($apiJson, true);
foreach($apiData['posts'] as $post) {
$item = array();
$item['author'] = 'Anonymous';
if ($post['senderData']['id'] !== false) {
$item['author'] = $post['senderData']['username'];
}
$item['uri'] = $this->getURI() . '/post/' . $post['id'];
$item['title'] = $this->ellipsisTitle($post['comment']);
$item['content'] = $this->processContent($post);
$item['timestamp'] = $post['timestamp'];
$this->items[] = $item;
}
}
public function getURI() {
if (!is_null($this->getInput('username'))) {
return self::URI . '/' . $this->getInput('username');
}
return parent::getURI();
}
public function getName() {
if (!is_null($this->getInput('username'))) {
return $this->getInput('username') . ' - Curious Cat';
}
return parent::getName();
}
private function processContent($post) {
$author = 'Anonymous';
if ($post['senderData']['id'] !== false) {
$authorUrl = self::URI . '/' . $post['senderData']['username'];
$author = <<<EOD
<a href="{$authorUrl}">{$post['senderData']['username']}</a>
EOD;
}
$question = $this->formatUrls($post['comment']);
$answer = $this->formatUrls($post['reply']);
$content = <<<EOD
<p>{$author} asked:</p>
<blockquote>{$question}</blockquote><br/>
<p>{$post['addresseeData']['username']} answered:</p>
<blockquote>{$answer}</blockquote>
EOD;
return $content;
}
private function ellipsisTitle($text) {
$length = 150;
if (strlen($text) > $length) {
$text = explode('<br>', wordwrap($text, $length, '<br>'));
return $text[0] . '...';
}
return $text;
}
private function formatUrls($content) {
return preg_replace(
'/(http[s]{0,1}\:\/\/[a-zA-Z0-9.\/\?\&=\-_]{4,})/ims',
'<a target="_blank" href="$1" target="_blank">$1</a> ',
$content
);
}
}

View File

@ -4,7 +4,7 @@ class DailymotionBridge extends BridgeAbstract {
const MAINTAINER = 'mitsukarenai';
const NAME = 'Dailymotion Bridge';
const URI = 'https://www.dailymotion.com/';
const CACHE_TIMEOUT = 10800; // 3h
const CACHE_TIMEOUT = 3600; // 1h
const DESCRIPTION = 'Returns the 5 newest videos by username/playlist or search';
const PARAMETERS = array (
@ -27,74 +27,99 @@ class DailymotionBridge extends BridgeAbstract {
),
'pa' => array(
'name' => 'Page',
'type' => 'number'
'type' => 'number',
'defaultValue' => 1,
)
)
);
protected function getMetadata($id){
$metadata = array();
$html2 = getSimpleHTMLDOM(self::URI . 'video/' . $id);
if(!$html2) {
return $metadata;
}
private $feedName = '';
$metadata['title'] = $html2->find('meta[property=og:title]', 0)->getAttribute('content');
$metadata['timestamp'] = strtotime(
$html2->find('meta[property=video:release_date]', 0)->getAttribute('content')
);
$metadata['thumbnailUri'] = $html2->find('meta[property=og:image]', 0)->getAttribute('content');
$metadata['uri'] = $html2->find('meta[property=og:url]', 0)->getAttribute('content');
return $metadata;
}
private $apiUrl = 'https://api.dailymotion.com';
private $apiFields = 'created_time,description,id,owner.screenname,tags,thumbnail_url,title,url';
public function getIcon() {
return 'https://static1-ssl.dmcdn.net/images/neon/favicons/android-icon-36x36.png.vf806ca4ed0deed812';
}
public function collectData(){
$html = '';
$limit = 5;
$count = 0;
public function collectData() {
$html = getSimpleHTMLDOM($this->getURI())
or returnServerError('Could not request Dailymotion.');
if ($this->queriedContext === 'By username' || $this->queriedContext === 'By playlist id') {
foreach($html->find('div.media a.preview_link') as $element) {
if($count < $limit) {
$apiJson = getContents($this->getApiUrl())
or returnServerError('Could not request: ' . $this->getApiUrl());
$apiData = json_decode($apiJson, true);
$this->feedName = $this->getPlaylistTitle($this->getInput('p'));
foreach ($apiData['list'] as $apiItem) {
$item = array();
$item['uri'] = $apiItem['url'];
$item['uid'] = $apiItem['id'];
$item['title'] = $apiItem['title'];
$item['timestamp'] = $apiItem['created_time'];
$item['author'] = $apiItem['owner.screenname'];
$item['content'] = '<p><a href="' . $apiItem['url'] . '">
<img src="' . $apiItem['thumbnail_url'] . '"></a></p><p>' . $apiItem['description'] . '</p>';
$item['categories'] = $apiItem['tags'];
$item['enclosures'][] = $apiItem['thumbnail_url'];
$this->items[] = $item;
}
}
if ($this->queriedContext === 'From search results') {
$html = getSimpleHTMLDOM($this->getURI())
or returnServerError('Could not request Dailymotion.');
foreach($html->find('div.media a.preview_link') as $element) {
$item = array();
$item['id'] = str_replace('/video/', '', strtok($element->href, '_'));
$metadata = $this->getMetadata($item['id']);
if(empty($metadata)) {
continue;
}
$item['uri'] = $metadata['uri'];
$item['title'] = $metadata['title'];
$item['timestamp'] = $metadata['timestamp'];
$item['content'] = '<a href="'
. $item['uri']
. '"><img src="'
. $metadata['thumbnailUri']
. '" /></a><br><a href="'
. $item['uri']
. '">'
. $item['title']
. '</a>';
. $item['uri']
. '"><img src="'
. $metadata['thumbnailUri']
. '" /></a><br><a href="'
. $item['uri']
. '">'
. $item['title']
. '</a>';
$this->items[] = $item;
$count++;
if (count($this->items) >= 5) {
break;
}
}
}
}
public function getName(){
public function getName() {
switch($this->queriedContext) {
case 'By username':
$specific = $this->getInput('u');
break;
case 'By playlist id':
$specific = strtok($this->getInput('p'), '_');
if ($this->feedName) {
$specific = $this->feedName;
}
break;
case 'From search results':
$specific = $this->getInput('s');
@ -102,26 +127,77 @@ class DailymotionBridge extends BridgeAbstract {
default: return parent::getName();
}
return $specific . ' : Dailymotion Bridge';
return $specific . ' : Dailymotion';
}
public function getURI(){
$uri = self::URI;
switch($this->queriedContext) {
case 'By username':
$uri .= 'user/' . urlencode($this->getInput('u')) . '/1';
$uri .= 'user/' . urlencode($this->getInput('u'));
break;
case 'By playlist id':
$uri .= 'playlist/' . urlencode(strtok($this->getInput('p'), '_'));
break;
case 'From search results':
$uri .= 'search/' . urlencode($this->getInput('s'));
if($this->getInput('pa')) {
$uri .= '/' . $this->getInput('pa');
if(!is_null($this->getInput('pa'))) {
$pa = $this->getInput('pa');
if ($this->getInput('pa') < 1) {
$pa = 1;
}
$uri .= '/' . $pa;
}
break;
default: return parent::getURI();
}
return $uri;
}
private function getMetadata($id) {
$metadata = array();
$html = getSimpleHTMLDOM(self::URI . 'video/' . $id);
if(!$html) {
return $metadata;
}
$metadata['title'] = $html->find('meta[property=og:title]', 0)->getAttribute('content');
$metadata['timestamp'] = strtotime(
$html->find('meta[property=video:release_date]', 0)->getAttribute('content')
);
$metadata['thumbnailUri'] = $html->find('meta[property=og:image]', 0)->getAttribute('content');
$metadata['uri'] = $html->find('meta[property=og:url]', 0)->getAttribute('content');
return $metadata;
}
private function getPlaylistTitle($id) {
$title = '';
$url = self::URI . 'playlist/' . $id;
$html = getSimpleHTMLDOM($url)
or returnServerError('Could not request: ' . $url);
$title = $html->find('meta[property=og:title]', 0)->getAttribute('content');
return $title;
}
private function getApiUrl() {
switch($this->queriedContext) {
case 'By username':
return $this->apiUrl . '/user/' . $this->getInput('u')
. '/videos?fields=' . urlencode($this->apiFields) . '&availability=1&sort=recent&limit=5';
break;
case 'By playlist id':
return $this->apiUrl . '/playlist/' . $this->getInput('p')
. '/videos?fields=' . urlencode($this->apiFields) . '&limit=5';
break;
}
}
}

View File

@ -40,7 +40,7 @@ class DanbooruBridge extends BridgeAbstract {
defaultLinkTo($element, $this->getURI());
$item = array();
$item['uri'] = $element->find('a', 0)->href;
$item['uri'] = html_entity_decode($element->find('a', 0)->href);
$item['postid'] = (int)preg_replace('/[^0-9]/', '', $element->getAttribute(static::IDATTRIBUTE));
$item['timestamp'] = time();
$thumbnailUri = $element->find('img', 0)->src;

View File

@ -0,0 +1,79 @@
<?php
class DarkReadingBridge extends FeedExpander {
const MAINTAINER = 'ORelio';
const NAME = 'Dark Reading Bridge';
const URI = 'https://www.darkreading.com/';
const DESCRIPTION = 'Returns the newest articles from Dark Reading';
const PARAMETERS = array( array(
'feed' => array(
'name' => 'Feed',
'type' => 'list',
'values' => array(
'All Dark Reading Stories' => '000_AllArticles',
'Attacks/Breaches' => '644_Attacks/Breaches',
'Application Security' => '645_Application%20Security',
'Database Security' => '646_Database%20Security',
'Cloud' => '647_Cloud',
'Endpoint' => '648_Endpoint',
'Authentication' => '649_Authentication',
'Privacy' => '650_Privacy',
'Mobile' => '651_Mobile',
'Perimeter' => '652_Perimeter',
'Risk' => '653_Risk',
'Compliance' => '654_Compliance',
'Operations' => '655_Operations',
'Careers and People' => '656_Careers%20and%20People',
'Identity and Access Management' => '657_Identity%20and%20Access%20Management',
'Analytics' => '658_Analytics',
'Threat Intelligence' => '659_Threat%20Intelligence',
'Security Monitoring' => '660_Security%20Monitoring',
'Vulnerabilities / Threats' => '661_Vulnerabilities%20/%20Threats',
'Advanced Threats' => '662_Advanced%20Threats',
'Insider Threats' => '663_Insider%20Threats',
'Vulnerability Management' => '664_Vulnerability%20Management',
)
)
));
public function collectData(){
$feed = $this->getInput('feed');
$feed_splitted = explode('_', $feed);
$feed_id = $feed_splitted[0];
$feed_name = $feed_splitted[1];
if(empty($feed) || !ctype_digit($feed_id) || !preg_match('/[A-Za-z%20\/]/', $feed_name)) {
returnClientError('Invalid feed, please check the "feed" parameter.');
}
$feed_url = $this->getURI() . 'rss_simple.asp';
if ($feed_id != '000') {
$feed_url .= '?f_n=' . $feed_id . '&f_ln=' . $feed_name;
}
$this->collectExpandableDatas($feed_url);
}
protected function parseItem($newsItem){
$item = parent::parseItem($newsItem);
$article = getSimpleHTMLDOMCached($item['uri'])
or returnServerError('Could not request Dark Reading: ' . $item['uri']);
$item['content'] = $this->extractArticleContent($article);
$item['enclosures'] = array(); //remove author profile picture
return $item;
}
private function extractArticleContent($article){
$content = $article->find('div#article-main', 0)->innertext;
foreach (array(
'<div class="divsplitter',
'<div style="float: left; margin-right: 2px;',
'<div class="more-insights',
'<div id="more-insights',
) as $div_start) {
$content = stripRecursiveHTMLSection($content, 'div', $div_start);
}
$content = stripWithDelimiters($content, '<h1 ', '</h1>');
return $content;
}
}

View File

@ -0,0 +1,27 @@
<?php
class DavesTrailerPageBridge extends BridgeAbstract {
const MAINTAINER = 'johnnygroovy';
const NAME = 'Daves Trailer Page Bridge';
const URI = 'https://www.davestrailerpage.co.uk/';
const DESCRIPTION = 'Last trailers in HD thanks to Dave.';
public function collectData(){
$html = getSimpleHTMLDOM(static::URI)
or returnClientError('No results for this query.');
foreach ($html->find('tr[!align]') as $tr) {
$item = array();
// title
$item['title'] = $tr->find('td', 0)->find('b', 0)->plaintext;
// content
$item['content'] = $tr->find('ul', 1);
// uri
$item['uri'] = $tr->find('a', 3)->getAttribute('href');
$this->items[] = $item;
}
}
}

View File

@ -1145,7 +1145,7 @@ class PepperBridgeAbstract extends BridgeAbstract {
} else {
foreach ($list as $deal) {
$item = array();
$item['uri'] = $deal->find('div[class=threadGrid-title]', 0)->find('a', 0)->href;
$item['uri'] = $deal->find('div[class*=threadGrid-title]', 0)->find('a', 0)->href;
$item['title'] = $deal->find('a[class*=' . $selectorLink . ']', 0
)->plaintext;
$item['author'] = $deal->find('span.thread-username', 0)->plaintext;

View File

@ -1,169 +0,0 @@
<?php
class DemonoidBridge extends BridgeAbstract {
const MAINTAINER = 'metaMMA';
const NAME = 'Demonoid';
const URI = 'https://www.demonoid.pw/';
const DESCRIPTION = 'Returns results from search';
const PARAMETERS = array(
'Keywords' => array(
'q' => array(
'name' => 'keywords',
'exampleValue' => 'keyword1 keyword2…',
'required' => true,
),
'category' => array(
'name' => 'Category',
'type' => 'list',
'values' => array(
'All' => 0,
'Movies' => 1,
'Music' => 2,
'TV' => 3,
'Games' => 4,
'Applications' => 5,
'Pictures' => 8,
'Anime' => 9,
'Comics' => 10,
'Books' => 11,
'Audiobooks' => 17
)
)
),
'Category Only' => array(
'catOnly' => array(
'name' => 'Category',
'type' => 'list',
'values' => array(
'All' => 0,
'Movies' => 1,
'Music' => 2,
'TV' => 3,
'Games' => 4,
'Applications' => 5,
'Pictures' => 8,
'Anime' => 9,
'Comics' => 10,
'Books' => 11,
'Audiobooks' => 17
)
)
),
'User ID' => array(
'userid' => array(
'name' => 'user id',
'exampleValue' => '00000',
'required' => true,
'type' => 'number'
),
'category' => array(
'name' => 'Category',
'type' => 'list',
'values' => array(
'All' => 0,
'Movies' => 1,
'Music' => 2,
'TV' => 3,
'Games' => 4,
'Applications' => 5,
'Pictures' => 8,
'Anime' => 9,
'Comics' => 10,
'Books' => 11,
'Audiobooks' => 17
)
)
)
);
public function collectData() {
if(!empty($this->getInput('q'))) {
$html = getSimpleHTMLDOM(
self::URI .
'files/?category=' .
rawurlencode($this->getInput('category')) .
'&subcategory=All&quality=All&seeded=2&external=2&query=' .
urlencode($this->getInput('q')) .
'&uid=0&sort='
) or returnServerError('Could not request Demonoid.');
} elseif(!empty($this->getInput('catOnly'))) {
$html = getSimpleHTMLDOM(
self::URI .
'files/?uid=0&category=' .
rawurlencode($this->getInput('catOnly')) .
'&subcategory=0&language=0&seeded=2&quality=0&query=&sort='
) or returnServerError('Could not request Demonoid.');
} elseif(!empty($this->getInput('userid'))) {
$html = getSimpleHTMLDOM(
self::URI .
'files/?uid=' .
rawurlencode($this->getInput('userid')) .
'&seeded=2'
) or returnServerError('Could not request Demonoid.');
} else {
returnServerError('Invalid parameters !');
}
if(preg_match('~No torrents found~', $html)) {
return;
}
$table = $html->find('td[class=ctable_content_no_pad]', 0);
$cursorCount = 4;
$elementCount = 0;
while($elementCount != 40) {
$elementCount++;
$currentElement = $table->find('tr', $cursorCount);
if(preg_match('~items total~', $currentElement)) {
break;
}
$item = array();
//Do we have a date ?
if(preg_match('~Added.*?(.*)~', $currentElement->plaintext, $dateStr)) {
if(preg_match('~today~', $dateStr[0])) {
date_default_timezone_set('UTC');
$timestamp = mktime(0, 0, 0, gmdate('n'), gmdate('j'), gmdate('Y'));
} else {
preg_match('~(?<=ed on ).*\d+~', $currentElement->plaintext, $fullDateStr);
date_default_timezone_set('UTC');
$dateObj = strptime($fullDateStr[0], '%A, %b %d, %Y');
$timestamp = mktime(0, 0, 0, $dateObj['tm_mon'] + 1, $dateObj['tm_mday'], 1900 + $dateObj['tm_year']);
}
$cursorCount++;
}
$content = $table->find('tr', $cursorCount)->find('a', 1);
$cursorCount++;
$torrentInfo = $table->find('tr', $cursorCount);
$item['timestamp'] = $timestamp;
$item['title'] = $content->plaintext;
$item['id'] = self::URI . $content->href;
$item['uri'] = self::URI . $content->href;
$item['author'] = $torrentInfo->find('a[class=user]', 0)->plaintext;
$item['seeders'] = $torrentInfo->find('font[class=green]', 0)->plaintext;
$item['leechers'] = $torrentInfo->find('font[class=red]', 0)->plaintext;
$item['size'] = $torrentInfo->find('td', 3)->plaintext;
$item['content'] = 'Uploaded by ' . $item['author']
. ' , Size ' . $item['size']
. '<br>seeders: '
. $item['seeders']
. ' | leechers: '
. $item['leechers']
. '<br><a href="'
. $item['id']
. '">info page</a>';
$this->items[] = $item;
$cursorCount++;
}
}
}

View File

@ -116,6 +116,12 @@ class DesoutterBridge extends BridgeAbstract {
'name' => 'Load full articles',
'type' => 'checkbox',
'title' => 'Enable to load the full article for each item'
),
'limit' => array(
'name' => 'Limit',
'type' => 'number',
'defaultValue' => 3,
'title' => "Maximum number of items to return in the feed.\n0 = unlimited"
)
)
);
@ -156,19 +162,23 @@ class DesoutterBridge extends BridgeAbstract {
$this->title = html_entity_decode($html->find('title', 0)->plaintext, ENT_QUOTES);
$limit = $this->getInput('limit') ?: 0;
foreach($html->find('article') as $article) {
$item = array();
$item['uri'] = $article->find('[itemprop="name"]', 0)->href;
$item['title'] = $article->find('[itemprop="name"]', 0)->title;
$item['uri'] = $article->find('a', 0)->href;
$item['title'] = $article->find('a[title]', 0)->title;
if($this->getInput('full')) {
$item['content'] = $this->getFullNewsArticle($item['uri']);
} else {
$item['content'] = $article->find('[itemprop="description"]', 0)->plaintext;
$item['content'] = $article->find('div.tile-body p', 0)->plaintext;
}
$this->items[] = $item;
if ($limit > 0 && count($this->items) >= $limit) break;
}
}

View File

@ -0,0 +1,60 @@
<?php
class DiarioDoAlentejoBridge extends BridgeAbstract {
const MAINTAINER = 'somini';
const NAME = 'Diário do Alentejo';
const URI = 'https://www.diariodoalentejo.pt';
const DESCRIPTION = 'Semanário Regionalista Independente';
const CACHE_TIMEOUT = 28800; // 8h
/* This is used to hack around obtaining a timestamp. It's just a list of Month names in Portuguese ... */
const PT_MONTH_NAMES = array(
'janeiro',
'fevereiro',
'março',
'abril',
'maio',
'junho',
'julho',
'agosto',
'setembro',
'outubro',
'novembro',
'dezembro');
public function getIcon() {
return 'https://www.diariodoalentejo.pt/images/favicon/apple-touch-icon.png';
}
public function collectData(){
/* This is slow as molasses (>30s!), keep the cache timeout high to avoid killing the host */
$html = getSimpleHTMLDOMCached($this->getURI() . '/pt/noticias-listagem.aspx')
or returnServerError('Could not load content');
foreach($html->find('.list_news .item') as $element) {
$item = array();
$item_link = $element->find('.body h2.title a', 0);
/* Another broken URL, see also `bridges/ComboiosDePortugalBridge.php` */
$item['uri'] = self::URI . implode('/', array_map('urlencode', explode('/', $item_link->href)));
$item['title'] = $item_link->innertext;
$item['timestamp'] = str_ireplace(
array_map(function($name) { return ' ' . $name . ' '; }, self::PT_MONTH_NAMES),
array_map(function($num) { return sprintf('-%02d-', $num); }, range(1, sizeof(self::PT_MONTH_NAMES))),
$element->find('span.date', 0)->innertext);
/* Fix the Image URL */
$item_image = $element->find('img.thumb', 0);
$item_image->src = preg_replace('/.*&img=([^&]+).*/', '\1', $item_image->getAttribute('data-src'));
/* Content: */
/* - Image */
/* - Category */
$content = $item_image .
'<center>' . $element->find('a.category', 0) . '</center>';
$item['content'] = defaultLinkTo($content, self::URI);
$this->items[] = $item;
}
}
}

View File

@ -1,9 +0,0 @@
<?php
require_once('Shimmie2Bridge.php');
class DollbooruBridge extends Shimmie2Bridge {
const MAINTAINER = 'mitsukarenai';
const NAME = 'Dollbooru';
const URI = 'http://dollbooru.org/';
const DESCRIPTION = 'Returns images from given page';
}

File diff suppressed because it is too large Load Diff

View File

@ -19,7 +19,7 @@ favicon-63b2904a073c89b52b19aa08cebc16a154bcf83fee8ecc6439968b1e6db569c7.ico';
$json = $this->loadEmbeddedJsonData($html);
foreach($html->find('li[id^="screenshot-"]') as $shot) {
$item = [];
$item = array();
$additional_data = $this->findJsonForShot($shot, $json);
if ($additional_data === null) {
@ -38,14 +38,14 @@ favicon-63b2904a073c89b52b19aa08cebc16a154bcf83fee8ecc6439968b1e6db569c7.ico';
$preview_path = $shot->find('picture source', 0)->attr['srcset'];
$item['content'] .= $this->getImageTag($preview_path, $item['title']);
$item['enclosures'] = [$this->getFullSizeImagePath($preview_path)];
$item['enclosures'] = array($this->getFullSizeImagePath($preview_path));
$this->items[] = $item;
}
}
private function loadEmbeddedJsonData($html){
$json = [];
$json = array();
$scripts = $html->find('script');
foreach($scripts as $script) {

View File

@ -40,7 +40,7 @@ class EconomistBridge extends BridgeAbstract {
if ($nextprev)
$nextprev->outertext = '';
$section = [ $article->find('h3[itemprop="articleSection"]', 0)->plaintext ];
$section = array( $article->find('h3[itemprop="articleSection"]', 0)->plaintext );
$item = array();
$item['title'] = $header->find('span', 0)->innertext . ': '

View File

@ -47,5 +47,8 @@ class EliteDangerousGalnetBridge extends BridgeAbstract {
$this->items[] = $item;
}
//Remove duplicates that sometimes show up on the website
$this->items = array_unique($this->items, SORT_REGULAR);
}
}

View File

@ -95,7 +95,7 @@ class ElloBridge extends BridgeAbstract {
private function getEnclosures($post, $postData) {
$assets = [];
$assets = array();
foreach($post->links->assets as $asset) {
foreach($postData->linked->assets as $assetLink) {
if($asset == $assetLink->id) {
@ -120,9 +120,11 @@ class ElloBridge extends BridgeAbstract {
}
private function getAPIKey() {
$cache = Cache::create(Configuration::getConfig('cache', 'type'));
$cacheFac = new CacheFactory();
$cacheFac->setWorkingDir(PATH_LIB_CACHES);
$cache = $cacheFac->create(Configuration::getConfig('cache', 'type'));
$cache->setScope(get_called_class());
$cache->setKey(['key']);
$cache->setKey(array('key'));
$key = $cache->loadData();
if($key == null) {

View File

@ -3,7 +3,7 @@ class ElsevierBridge extends BridgeAbstract {
const MAINTAINER = 'Pierre Mazière';
const NAME = 'Elsevier journals recent articles';
const URI = 'http://www.journals.elsevier.com/';
const URI = 'https://www.journals.elsevier.com/';
const CACHE_TIMEOUT = 43200; //12h
const DESCRIPTION = 'Returns the recent articles published in Elsevier journals';

View File

@ -0,0 +1,26 @@
<?php
class EngadgetBridge extends FeedExpander {
const MAINTAINER = 'IceWreck';
const NAME = 'Engadget Bridge';
const URI = 'https://www.engadget.com/';
const CACHE_TIMEOUT = 3600;
const DESCRIPTION = 'Article content for Engadget.';
public function collectData(){
$this->collectExpandableDatas(static::URI . 'rss.xml', 15);
}
protected function parseItem($newsItem){
$item = parent::parseItem($newsItem);
// $articlePage gets the entire page's contents
$articlePage = getSimpleHTMLDOM($newsItem->link);
// figure contain's the main article image
$article = $articlePage->find('figure', 0);
// .article-text has the actual article
foreach($articlePage->find('.article-text') as $element)
$article = $article . $element;
$item['content'] = $article;
return $item;
}
}

View File

@ -0,0 +1,70 @@
<?php
class EsquerdaNetBridge extends FeedExpander {
const MAINTAINER = 'somini';
const NAME = 'Esquerda.net';
const URI = 'https://www.esquerda.net';
const DESCRIPTION = 'Esquerda.net';
const PARAMETERS = array(
array(
'feed' => array(
'name' => 'Feed',
'type' => 'list',
'defaultValue' => 'Geral',
'values' => array(
'Geral' => 'geral',
'Dossier' => 'artigos-dossier',
'Vídeo' => 'video',
'Opinião' => 'opinioes',
'Rádio' => 'radio',
)
)
)
);
public function getURI() {
$type = $this->getInput('feed');
return self::URI . '/rss/' . $type;
}
public function getIcon() {
return 'https://www.esquerda.net/sites/default/files/favicon_0.ico';
}
public function collectData(){
parent::collectExpandableDatas($this->getURI());
}
protected function parseItem($newsItem){
# Fix Publish date
$badDate = $newsItem->pubDate;
preg_match('|(?P<day>\d\d)/(?P<month>\d\d)/(?P<year>\d\d\d\d) - (?P<hour>\d\d):(?P<minute>\d\d)|', $badDate, $d);
$newsItem->pubDate = sprintf('%s-%s-%sT%s:%s', $d['year'], $d['month'], $d['day'], $d['hour'], $d['minute']);
$item = parent::parseItem($newsItem);
# Include all the content
$uri = $item['uri'];
$html = getSimpleHTMLDOMCached($uri)
or returnServerError('Could not load content for ' . $uri);
$content = $html->find('div#content div.content', 0);
## Fix author
$authorHTML = $html->find('.field-name-field-op-author a', 0);
if ($authorHTML) {
$item['author'] = $authorHTML->innertext;
$authorHTML->remove();
}
## Remove crap
$content->find('.field-name-addtoany', 0)->remove();
## Fix links
$content = defaultLinkTo($content, self::URI);
## Fix Images
foreach($content->find('img') as $img) {
$altSrc = $img->getAttribute('data-src');
if ($altSrc) {
$img->setAttribute('src', $altSrc);
}
$img->width = null;
$img->height = null;
}
$item['content'] = $content;
return $item;
}
}

View File

@ -1,7 +1,7 @@
<?php
class ExtremeDownloadBridge extends BridgeAbstract {
const NAME = 'Extreme Download';
const URI = 'https://ww1.extreme-d0wn.com/';
const URI = 'https://wvw.extreme-down.xyz/';
const DESCRIPTION = 'Suivi de série sur Extreme Download';
const MAINTAINER = 'sysadminstory';
const PARAMETERS = array(

View File

@ -2,7 +2,7 @@
class FB2Bridge extends BridgeAbstract {
const MAINTAINER = 'teromene';
const NAME = 'Facebook Alternate';
const NAME = 'Facebook Bridge | Touch Site';
const URI = 'https://www.facebook.com/';
const CACHE_TIMEOUT = 1000;
const DESCRIPTION = 'Input a page title or a profile log. For a profile log,
@ -12,7 +12,12 @@ class FB2Bridge extends BridgeAbstract {
'u' => array(
'name' => 'Username',
'required' => true
)
),
'abbrev_name' => array(
'name' => 'Abbreviate author name in title',
'type' => 'checkbox',
'defaultValue' => true,
),
));
public function getIcon() {
@ -102,12 +107,12 @@ EOD
else
$timestamp = 0;
$item['uri'] = html_entity_decode('http://touch.facebook.com'
$item['uri'] = html_entity_decode('https://touch.facebook.com'
. $content->find("div[class='_52jc _5qc4 _78cz _24u0 _36xo']", 0)->find('a', 0)->getAttribute('href'), ENT_QUOTES);
//Decode images
$imagecleaned = preg_replace_callback('/<i [^>]* style="[^"]*url\(\'(.*?)\'\).*?><\/i>/m', function ($matches) {
return "<img src='" . str_replace(['\\3a ', '\\3d ', '\\26 '], [':', '=', '&'], $matches[1]) . "' />";
return "<img src='" . str_replace(array('\\3a ', '\\3d ', '\\26 '), array(':', '=', '&'), $matches[1]) . "' />";
}, $content);
$content = str_get_html($imagecleaned);
@ -159,7 +164,11 @@ EOD
$content = preg_replace('/<img src=\'.*?safe_image\.php.*?\' \/>/m', '', $content);
//Remove the double section tags
$content = str_replace(['<section><section>', '</section></section>'], ['<section>', '</section>'], $content);
$content = str_replace(
array('<section><section>', '</section></section>'),
array('<section>', '</section>'),
$content
);
//Move the section tag link upper, if it is down
$content = str_get_html($content);
@ -182,8 +191,10 @@ EOD
$item['content'] = html_entity_decode($content, ENT_QUOTES);
$title = $author;
if (strlen($title) > 24)
$title = substr($title, 0, strpos(wordwrap($title, 24), "\n")) . '...';
if ($this->getInput('abbrev_name') === true) {
if (strlen($title) > 24)
$title = substr($title, 0, strpos(wordwrap($title, 24), "\n")) . '...';
}
$title = $title . ' | ' . strip_tags($content);
if (strlen($title) > 64)
$title = substr($title, 0, strpos(wordwrap($title, 64), "\n")) . '...';
@ -281,10 +292,20 @@ EOD
}
public function getName(){
return (isset($this->name) ? $this->name . ' - ' : '') . 'Facebook Bridge';
$username = $this->getInput('u');
if (isset($username)) {
return $this->getInput('u') . ' | Facebook';
} else {
return self::NAME;
}
}
public function getURI(){
return 'http://facebook.com';
$username = $this->getInput('u');
if (isset($username)) {
return 'https://facebook.com/' . $this->getInput('u') . '/posts';
} else {
return self::URI;
}
}
}

View File

@ -0,0 +1,36 @@
<?php
class FabriceBellardBridge extends BridgeAbstract {
const NAME = 'Fabrice Bellard';
const URI = 'https://bellard.org/';
const DESCRIPTION = "Fabrice Bellard's Home Page";
const MAINTAINER = 'somini';
public function collectData() {
$html = getSimpleHTMLDOM(self::URI)
or returnServerError('Could not load content');
foreach ($html->find('p') as $obj) {
$item = array();
$html = defaultLinkTo($html, $this->getURI());
$links = $obj->find('a');
if (count($links) > 0) {
$link_uri = $links[0]->href;
} else {
$link_uri = $this->getURI();
}
/* try to make sure the link is valid */
if ($link_uri[-1] !== '/' && strpos($link_uri, '/') === false) {
$link_uri = $link_uri . '/';
}
$item['title'] = strip_tags($obj->innertext);
$item['uri'] = $link_uri;
$item['content'] = $obj->innertext;
$this->items[] = $item;
}
}
}

View File

@ -2,7 +2,7 @@
class FacebookBridge extends BridgeAbstract {
const MAINTAINER = 'teromene, logmanoriginal';
const NAME = 'Facebook Bridge';
const NAME = 'Facebook Bridge | Main Site';
const URI = 'https://www.facebook.com/';
const CACHE_TIMEOUT = 300; // 5min
const DESCRIPTION = 'Input a page title or a profile log. For a profile log,
@ -66,14 +66,13 @@ class FacebookBridge extends BridgeAbstract {
case 'User':
if(!empty($this->authorName)) {
return isset($this->extraInfos['name']) ? $this->extraInfos['name'] : $this->authorName
. ' - ' . static::NAME;
return isset($this->extraInfos['name']) ? $this->extraInfos['name'] : $this->authorName;
}
break;
case 'Group':
if(!empty($this->groupName)) {
return $this->groupName . ' - ' . static::NAME;
return $this->groupName;
}
break;
@ -82,6 +81,34 @@ class FacebookBridge extends BridgeAbstract {
return parent::getName();
}
public function detectParameters($url){
$params = array();
// By profile
$regex = '/^(https?:\/\/)?(www\.)?facebook\.com\/profile\.php\?id\=([^\/?&\n]+)?(.*)/';
if(preg_match($regex, $url, $matches) > 0) {
$params['u'] = urldecode($matches[3]);
return $params;
}
// By group
$regex = '/^(https?:\/\/)?(www\.)?facebook\.com\/groups\/([^\/?\n]+)?(.*)/';
if(preg_match($regex, $url, $matches) > 0) {
$params['g'] = urldecode($matches[3]);
return $params;
}
// By username
$regex = '/^(https?:\/\/)?(www\.)?facebook\.com\/([^\/?\n]+)/';
if(preg_match($regex, $url, $matches) > 0) {
$params['u'] = urldecode($matches[3]);
return $params;
}
return null;
}
public function getURI() {
$uri = self::URI;
@ -142,7 +169,11 @@ class FacebookBridge extends BridgeAbstract {
private function collectGroupData() {
$header = array('Accept-Language: ' . getEnv('HTTP_ACCEPT_LANGUAGE') . "\r\n");
if(getEnv('HTTP_ACCEPT_LANGUAGE')) {
$header = array('Accept-Language: ' . getEnv('HTTP_ACCEPT_LANGUAGE'));
} else {
$header = array();
}
$html = getSimpleHTMLDOM($this->getURI(), $header)
or returnServerError('Failed loading facebook page: ' . $this->getURI());
@ -505,7 +536,11 @@ EOD;
// Retrieve page contents
if(is_null($html)) {
$header = array('Accept-Language: ' . getEnv('HTTP_ACCEPT_LANGUAGE'));
if(getEnv('HTTP_ACCEPT_LANGUAGE')) {
$header = array('Accept-Language: ' . getEnv('HTTP_ACCEPT_LANGUAGE'));
} else {
$header = array();
}
$html = getSimpleHTMLDOM($this->getURI(), $header)
or returnServerError('No results for this query.');
@ -580,6 +615,8 @@ EOD;
'._5mly', // Remove embedded videos (the preview image remains)
'._2ezg', // Remove "Views ..."
'.hidden_elem', // Remove hidden elements (they are hidden anyway)
'.timestampContent', // Remove relative timestamp
'._6spk', // Remove redundant separator
);
foreach($content_filters as $filter) {
@ -664,8 +701,15 @@ EOD;
$uri = $post->find('abbr')[0]->parent()->getAttribute('href');
if (false !== strpos($uri, '?')) {
$uri = substr($uri, 0, strpos($uri, '?'));
// Extract fbid and patch link
if (strpos($uri, '?') !== false) {
$query = substr($uri, strpos($uri, '?') + 1);
parse_str($query, $query_params);
if (isset($query_params['story_fbid'])) {
$uri = self::URI . $query_params['story_fbid'];
} else {
$uri = substr($uri, 0, strpos($uri, '?'));
}
}
//Build and add final item

164
bridges/FicbookBridge.php Normal file
View File

@ -0,0 +1,164 @@
<?php
class FicbookBridge extends BridgeAbstract {
const NAME = 'Ficbook Bridge';
const URI = 'https://ficbook.net/';
const DESCRIPTION = 'No description provided';
const MAINTAINER = 'logmanoriginal';
const PARAMETERS = array(
'Site News' => array(),
'Fiction Updates' => array(
'fiction_id' => array(
'name' => 'Fanfiction ID',
'type' => 'text',
'pattern' => '[0-9]+',
'required' => true,
'title' => 'Insert fanfiction ID',
'exampleValue' => '5783919',
),
'include_contents' => array(
'name' => 'Include contents',
'type' => 'checkbox',
'title' => 'Activate to include contents in the feed',
),
),
'Fiction Comments' => array(
'fiction_id' => array(
'name' => 'Fanfiction ID',
'type' => 'text',
'pattern' => '[0-9]+',
'required' => true,
'title' => 'Insert fanfiction ID',
'exampleValue' => '5783919',
),
),
);
public function getURI() {
switch($this->queriedContext) {
case 'Site News': {
// For some reason this is not HTTPS
return 'http://ficbook.net/sitenews';
}
case 'Fiction Updates': {
return self::URI
. 'readfic/'
. urlencode($this->getInput('fiction_id'));
}
case 'Fiction Comments': {
return self::URI
. 'readfic/'
. urlencode($this->getInput('fiction_id'))
. '/comments#content';
}
default: return parent::getURI();
}
}
public function collectData() {
$header = array('Accept-Language: en-US');
$html = getSimpleHTMLDOM($this->getURI(), $header)
or returnServerError('Could not request ' . $this->getURI());
$html = defaultLinkTo($html, self::URI);
switch($this->queriedContext) {
case 'Site News': return $this->collectSiteNews($html);
case 'Fiction Updates': return $this->collectUpdatesData($html);
case 'Fiction Comments': return $this->collectCommentsData($html);
}
}
private function collectSiteNews($html) {
foreach($html->find('.news_view') as $news) {
$this->items[] = array(
'title' => $news->find('h1.title', 0)->plaintext,
'timestamp' => strtotime($this->fixDate($news->find('span[title]', 0)->title)),
'content' => $news->find('.news_text', 0),
);
}
}
private function collectCommentsData($html) {
foreach($html->find('article.post') as $article) {
$this->items[] = array(
'uri' => $article->find('.comment_link_to_fic > a', 0)->href,
'title' => $article->find('.comment_author', 0)->plaintext,
'author' => $article->find('.comment_author', 0)->plaintext,
'timestamp' => strtotime($this->fixDate($article->find('time[datetime]', 0)->datetime)),
'content' => $article->find('.comment_message', 0),
'enclosures' => array($article->find('img', 0)->src),
);
}
}
private function collectUpdatesData($html) {
foreach($html->find('ul.table-of-contents > li') as $chapter) {
$item = array(
'uri' => $chapter->find('a', 0)->href,
'title' => $chapter->find('a', 0)->plaintext,
'timestamp' => strtotime($this->fixDate($chapter->find('span[title]', 0)->title)),
);
if($this->getInput('include_contents')) {
$content = getSimpleHTMLDOMCached($item['uri']);
$item['content'] = $content->find('#content', 0);
}
$this->items[] = $item;
// Sort by time, descending
usort($this->items, function($a, $b){ return $b['timestamp'] - $a['timestamp']; });
}
}
private function fixDate($date) {
// FIXME: This list was generated using Google tranlator. Someone who
// actually knows russian should check this list! Please keep in mind
// that month names must match exactly the names returned by Ficbook.
$ru_month = array(
'января',
'февраля',
'марта',
'апреля',
'мая',
'июня',
'июля',
'августа',
'Сентября',
'октября',
'Ноября',
'Декабря',
);
$en_month = array(
'January',
'February',
'March',
'April',
'May',
'June',
'July',
'August',
'September',
'October',
'November',
'December',
);
$fixed_date = str_replace($ru_month, $en_month, $date);
if($fixed_date === $date) {
Debug::log('Unable to fix date: ' . $date);
return null;
}
return $fixed_date;
}
}

View File

@ -62,11 +62,16 @@ class FindACrewBridge extends BridgeAbstract {
foreach ($annonces as $annonce) {
$item = array();
$img = parent::getURI() . $annonce->find('.lst-pic img', 0)->getAttribute('src');
$link = parent::getURI() . $annonce->find('.lst-ctrls a', 0)->href;
$htmlDetail = getSimpleHTMLDOMCached($link . '?mdl=2'); // add ?mdl=2 for xhr content not full html page
$img = parent::getURI() . $htmlDetail->find('img.img-responsive', 0)->getAttribute('src');
$item['title'] = $annonce->find('.lst-tags span', 0)->plaintext;
$item['uri'] = parent::getURI() . $annonce->find('.lst-ctrls a', 0)->href;
$content = $annonce->find('.lst-dtl', 0)->innertext;
$item['content'] = "<img src='$img' /><br>$content";
$item['uri'] = $link;
$content = $htmlDetail->find('.panel-body div.clearfix.row > div', 1)->innertext;
$content .= $htmlDetail->find('.panel-body > div', 1)->innertext;
$content = defaultLinkTo($content, parent::getURI());
$item['content'] = $content;
$item['enclosures'] = array($img);
$item['categories'] = array($annonce->find('.css_AccLocCur', 0)->plaintext);
$this->items[] = $item;

View File

@ -0,0 +1,27 @@
<?php
class FreeCodeCampBridge extends FeedExpander {
const MAINTAINER = 'IceWreck';
const NAME = 'FreeCodecamp Bridge';
const URI = 'https://www.freecodecamp.org';
const CACHE_TIMEOUT = 3600;
const DESCRIPTION = 'RSS feed for FreeCodeCamp';
// Freecodecamp removed their old full content rss feed and replaced it with one liner content.
public function collectData(){
$this->collectExpandableDatas('https://www.freecodecamp.org/news/rss/', 15);
}
protected function parseItem($newsItem){
$item = parent::parseItem($newsItem);
// $articlePage gets the entire page's contents
$articlePage = getSimpleHTMLDOM($newsItem->link);
// figure contain's the main article image
$article = $articlePage->find('figure', 0);
// the actual article
foreach($articlePage->find('.post-full-content') as $element)
$article = $article . $element;
$item['content'] = $article;
return $item;
}
}

View File

@ -0,0 +1,918 @@
<?php
class FurAffinityBridge extends BridgeAbstract {
const NAME = 'FurAffinity Bridge';
const URI = 'https://www.furaffinity.net';
const CACHE_TIMEOUT = 300; // 5min
const DESCRIPTION = 'Returns posts from various sections of FurAffinity';
const MAINTAINER = 'Roliga';
const PARAMETERS = array(
'Search' => array(
'q' => array(
'name' => 'Query',
'required' => true
),
'rating-general' => array(
'name' => 'General',
'type' => 'checkbox',
'defaultValue' => 'checked'
),
'rating-mature' => array(
'name' => 'Mature',
'type' => 'checkbox',
),
'rating-adult' => array(
'name' => 'Adult',
'type' => 'checkbox',
),
'range' => array(
'name' => 'Time range',
'type' => 'list',
'values' => array(
'A Day' => 'day',
'3 Days' => '3days',
'A Week' => 'week',
'A Month' => 'month',
'All time' => 'all'
),
'defaultValue' => 'all'
),
'type-art' => array(
'name' => 'Art',
'type' => 'checkbox',
'defaultValue' => 'checked'
),
'type-flash' => array(
'name' => 'Flash',
'type' => 'checkbox',
'defaultValue' => 'checked'
),
'type-photo' => array(
'name' => 'Photography',
'type' => 'checkbox',
'defaultValue' => 'checked'
),
'type-music' => array(
'name' => 'Music',
'type' => 'checkbox',
'defaultValue' => 'checked'
),
'type-story' => array(
'name' => 'Story',
'type' => 'checkbox',
'defaultValue' => 'checked'
),
'type-poetry' => array(
'name' => 'Poetry',
'type' => 'checkbox',
'defaultValue' => 'checked'
),
'mode' => array(
'name' => 'Match mode',
'type' => 'list',
'values' => array(
'All of the words' => 'all',
'Any of the words' => 'any',
'Extended' => 'extended'
),
'defaultValue' => 'extended'
),
'limit' => array(
'name' => 'Limit',
'type' => 'number',
'defaultValue' => 10,
'title' => 'Limit number of submissions to return. -1 for unlimited.'
),
'full' => array(
'name' => 'Full view',
'title' => 'Include description, tags, date and larger image in article. Uses more bandwidth.',
'type' => 'checkbox',
'defaultValue' => 'checked'
),
'cache' => array(
'name' => 'Cache submission pages',
'title' => 'Reduces requests to FA when Full view is enabled. Changes to submission details may be delayed.',
'type' => 'checkbox',
'defaultValue' => 'checked'
)
),
'Browse' => array(
'cat' => array(
'name' => 'Category',
'type' => 'list',
'values' => array(
'Visual Art' => array(
'All' => 1,
'Artwork (Digital)' => 2,
'Artwork (Traditional)' => 3,
'Cellshading' => 4,
'Crafting' => 5,
'Designs' => 6,
'Flash' => 7,
'Fursuiting' => 8,
'Icons' => 9,
'Mosaics' => 10,
'Photography' => 11,
'Sculpting' => 12
),
'Readable Art' => array(
'Story' => 13,
'Poetry' => 14,
'Prose' => 15
),
'Audio Art' => array(
'Music' => 16,
'Podcasts' => 17
),
'Downloadable' => array(
'Skins' => 18,
'Handhelds' => 19,
'Resources' => 20
),
'Other Stuff' => array(
'Adoptables' => 21,
'Auctions' => 22,
'Contests' => 23,
'Current Events' => 24,
'Desktops' => 25,
'Stockart' => 26,
'Screenshots' => 27,
'Scraps' => 28,
'Wallpaper' => 29,
'YCH / Sale' => 30,
'Other' => 31
)
),
'defaultValue' => 1
),
'atype' => array(
'name' => 'Type',
'type' => 'list',
'values' => array(
'General Things' => array(
'All' => 1,
'Abstract' => 2,
'Animal related (non-anthro)' => 3,
'Anime' => 4,
'Comics' => 5,
'Doodle' => 6,
'Fanart' => 7,
'Fantasy' => 8,
'Human' => 9,
'Portraits' => 10,
'Scenery' => 11,
'Still Life' => 12,
'Tutorials' => 13,
'Miscellaneous' => 14
),
'Fetish / Furry specialty' => array(
'Baby fur' => 101,
'Bondage' => 102,
'Digimon' => 103,
'Fat Furs' => 104,
'Fetish Other' => 105,
'Fursuit' => 106,
'Gore / Macabre Art' => 119,
'Hyper' => 107,
'Inflation' => 108,
'Macro / Micro' => 109,
'Muscle' => 110,
'My Little Pony / Brony' => 111,
'Paw' => 112,
'Pokemon' => 113,
'Pregnancy' => 114,
'Sonic' => 115,
'Transformation' => 116,
'Vore' => 117,
'Water Sports' => 118,
'General Furry Art' => 100
),
'Music' => array(
'Techno' => 201,
'Trance' => 202,
'House' => 203,
'90s' => 204,
'80s' => 205,
'70s' => 206,
'60s' => 207,
'Pre-60s' => 208,
'Classical' => 209,
'Game Music' => 210,
'Rock' => 211,
'Pop' => 212,
'Rap' => 213,
'Industrial' => 214,
'Other Music' => 200
)
),
'defaultValue' => 1
),
'species' => array(
'name' => 'Species',
'type' => 'list',
'values' => array(
'Unspecified / Any' => 1,
'Amphibian' => array(
'Frog' => 1001,
'Newt' => 1002,
'Salamander' => 1003,
'Amphibian (Other)' => 1000
),
'Aquatic' => array(
'Cephalopod' => 2001,
'Dolphin' => 2002,
'Fish' => 2005,
'Porpoise' => 2004,
'Seal' => 6068,
'Shark' => 2006,
'Whale' => 2003,
'Aquatic (Other)' => 2000
),
'Avian' => array(
'Corvid' => 3001,
'Crow' => 3002,
'Duck' => 3003,
'Eagle' => 3004,
'Falcon' => 3005,
'Goose' => 3006,
'Gryphon' => 3007,
'Hawk' => 3008,
'Owl' => 3009,
'Phoenix' => 3010,
'Swan' => 3011,
'Avian (Other)' => 3000
),
'Bears &amp; Ursines' => array(
'Bear' => 6002
),
'Camelids' => array(
'Camel' => 6074,
'Llama' => 6036
),
'Canines &amp; Lupines' => array(
'Coyote' => 6008,
'Doberman' => 6009,
'Dog' => 6010,
'Dingo' => 6011,
'German Shepherd' => 6012,
'Jackal' => 6013,
'Husky' => 6014,
'Wolf' => 6016,
'Canine (Other)' => 6017
),
'Cervines' => array(
'Cervine (Other)' => 6018
),
'Cows &amp; Bovines' => array(
'Antelope' => 6004,
'Cows' => 6003,
'Gazelle' => 6005,
'Goat' => 6006,
'Bovines (General)' => 6007
),
'Dragons' => array(
'Eastern Dragon' => 4001,
'Hydra' => 4002,
'Serpent' => 4003,
'Western Dragon' => 4004,
'Wyvern' => 4005,
'Dragon (Other)' => 4000
),
'Equestrians' => array(
'Donkey' => 6019,
'Horse' => 6034,
'Pony' => 6073,
'Zebra' => 6071
),
'Exotic &amp; Mythicals' => array(
'Argonian' => 5002,
'Chakat' => 5003,
'Chocobo' => 5004,
'Citra' => 5005,
'Crux' => 5006,
'Daemon' => 5007,
'Digimon' => 5008,
'Dracat' => 5009,
'Draenei' => 5010,
'Elf' => 5011,
'Gargoyle' => 5012,
'Iksar' => 5013,
'Kaiju/Monster' => 5015,
'Langurhali' => 5014,
'Moogle' => 5017,
'Naga' => 5016,
'Orc' => 5018,
'Pokemon' => 5019,
'Satyr' => 5020,
'Sergal' => 5021,
'Tanuki' => 5022,
'Unicorn' => 5023,
'Xenomorph' => 5024,
'Alien (Other)' => 5001,
'Exotic (Other)' => 5000
),
'Felines' => array(
'Domestic Cat' => 6020,
'Cheetah' => 6021,
'Cougar' => 6022,
'Jaguar' => 6023,
'Leopard' => 6024,
'Lion' => 6025,
'Lynx' => 6026,
'Ocelot' => 6027,
'Panther' => 6028,
'Tiger' => 6029,
'Feline (Other)' => 6030
),
'Insects' => array(
'Arachnid' => 8000,
'Mantid' => 8004,
'Scorpion' => 8005,
'Insect (Other)' => 8003
),
'Mammals (Other)' => array(
'Bat' => 6001,
'Giraffe' => 6031,
'Hedgehog' => 6032,
'Hippopotamus' => 6033,
'Hyena' => 6035,
'Panda' => 6052,
'Pig/Swine' => 6053,
'Rabbit/Hare' => 6059,
'Raccoon' => 6060,
'Red Panda' => 6062,
'Meerkat' => 6043,
'Mongoose' => 6044,
'Rhinoceros' => 6063,
'Mammals (Other)' => 6000
),
'Marsupials' => array(
'Opossum' => 6037,
'Kangaroo' => 6038,
'Koala' => 6039,
'Quoll' => 6040,
'Wallaby' => 6041,
'Marsupial (Other)' => 6042
),
'Mustelids' => array(
'Badger' => 6045,
'Ferret' => 6046,
'Mink' => 6048,
'Otter' => 6047,
'Skunk' => 6069,
'Weasel' => 6049,
'Mustelid (Other)' => 6051
),
'Primates' => array(
'Gorilla' => 6054,
'Human' => 6055,
'Lemur' => 6056,
'Monkey' => 6057,
'Primate (Other)' => 6058
),
'Reptillian' => array(
'Alligator &amp; Crocodile' => 7001,
'Gecko' => 7003,
'Iguana' => 7004,
'Lizard' => 7005,
'Snakes &amp; Serpents' => 7006,
'Turtle' => 7007,
'Reptilian (Other)' => 7000
),
'Rodents' => array(
'Beaver' => 6064,
'Mouse' => 6065,
'Rat' => 6061,
'Squirrel' => 6070,
'Rodent (Other)' => 6067
),
'Vulpines' => array(
'Fennec' => 6072,
'Fox' => 6075,
'Vulpine (Other)' => 6015
),
'Other' => array(
'Dinosaur' => 8001,
'Wolverine' => 6050
)
),
'defaultValue' => 1
),
'gender' => array(
'name' => 'Gender',
'type' => 'list',
'values' => array(
'Any' => 0,
'Male' => 2,
'Female' => 3,
'Herm' => 4,
'Transgender' => 5,
'Multiple characters' => 6,
'Other / Not Specified' => 7
),
'defaultValue' => 0
),
'rating_general' => array(
'name' => 'General',
'type' => 'checkbox',
'defaultValue' => 'checked'
),
'rating_mature' => array(
'name' => 'Mature',
'type' => 'checkbox',
),
'rating_adult' => array(
'name' => 'Adult',
'type' => 'checkbox',
),
'limit-browse' => array(
'name' => 'Limit',
'type' => 'number',
'required' => true,
'defaultValue' => 10,
'title' => 'Limit number of submissions to return. -1 for unlimited.'
),
'full' => array(
'name' => 'Full view',
'title' => 'Include description, tags, date and larger image in article. Uses more bandwidth.',
'type' => 'checkbox',
'defaultValue' => 'checked'
),
'cache' => array(
'name' => 'Cache submission pages',
'title' => 'Reduces requests to FA when Full view is enabled. Changes to submission details may be delayed.',
'type' => 'checkbox',
'defaultValue' => 'checked'
)
),
'Journals' => array(
'username-journals' => array(
'name' => 'Username',
'required' => true,
'title' => 'Lowercase username as seen in URLs'
),
'limit' => array(
'name' => 'Limit',
'type' => 'number',
'defaultValue' => -1,
'title' => 'Limit number of journals to return. -1 for unlimited.'
)
),
'Single Journal' => array(
'journal-id' => array(
'name' => 'Journal ID',
'required' => true,
'type' => 'number',
'title' => 'Number seen in journal URL'
)
),
'Gallery' => array(
'username-gallery' => array(
'name' => 'Username',
'required' => true,
'title' => 'Lowercase username as seen in URLs'
),
'limit' => array(
'name' => 'Limit',
'type' => 'number',
'defaultValue' => 10,
'title' => 'Limit number of submissions to return. -1 for unlimited.'
),
'full' => array(
'name' => 'Full view',
'title' => 'Include description, tags, date and larger image in article. Uses more bandwidth.',
'type' => 'checkbox',
'defaultValue' => 'checked'
),
'cache' => array(
'name' => 'Cache submission pages',
'title' => 'Reduces requests to FA when Full view is enabled. Changes to submission details may be delayed.',
'type' => 'checkbox',
'defaultValue' => 'checked'
)
),
'Scraps' => array(
'username-scraps' => array(
'name' => 'Username',
'required' => true,
'title' => 'Lowercase username as seen in URLs'
),
'limit' => array(
'name' => 'Limit',
'type' => 'number',
'defaultValue' => 10,
'title' => 'Limit number of submissions to return. -1 for unlimited.'
),
'full' => array(
'name' => 'Full view',
'title' => 'Include description, tags, date and larger image in article. Uses more bandwidth.',
'type' => 'checkbox',
'defaultValue' => 'checked'
),
'cache' => array(
'name' => 'Cache submission pages',
'title' => 'Reduces requests to FA when Full view is enabled. Changes to submission details may be delayed.',
'type' => 'checkbox',
'defaultValue' => 'checked'
)
),
'Favorites' => array(
'username-favorites' => array(
'name' => 'Username',
'required' => true,
'title' => 'Lowercase username as seen in URLs'
),
'limit' => array(
'name' => 'Limit',
'type' => 'number',
'defaultValue' => 10,
'title' => 'Limit number of submissions to return. -1 for unlimited.'
),
'full' => array(
'name' => 'Full view',
'title' => 'Include description, tags, date and larger image in article. Uses more bandwidth.',
'type' => 'checkbox',
'defaultValue' => 'checked'
),
'cache' => array(
'name' => 'Cache submission pages',
'title' => 'Reduces requests to FA when Full view is enabled. Changes to submission details may be delayed.',
'type' => 'checkbox',
'defaultValue' => 'checked'
)
),
'Gallery Folder' => array(
'username-folder' => array(
'name' => 'Username',
'required' => true,
'title' => 'Lowercase username as seen in URLs'
),
'folder-id' => array(
'name' => 'Folder ID',
'required' => true,
'type' => 'number',
'title' => 'Number seen in folder URL'
),
'limit' => array(
'name' => 'Limit',
'type' => 'number',
'defaultValue' => 10,
'title' => 'Limit number of submissions to return. -1 for unlimited.'
),
'full' => array(
'name' => 'Full view',
'title' => 'Include description, tags, date and larger image in article. Uses more bandwidth.',
'type' => 'checkbox',
'defaultValue' => 'checked'
),
'cache' => array(
'name' => 'Cache submission pages',
'title' => 'Reduces requests to FA when Full view is enabled. Changes to submission details may be delayed.',
'type' => 'checkbox',
'defaultValue' => 'checked'
)
)
);
/*
* This was aquired by creating a new user on FA then
* extracting the cookie from the browsers dev console.
*/
const FA_AUTH_COOKIE = 'b=4ce65691-b50f-4742-a990-bf28d6de16ee; a=ca6e4566-9d81-4263-9444-653b142e35f8';
public function detectParameters($url) {
$params = array();
// Single journal
$regex = '/^(https?:\/\/)?(www\.)?furaffinity.net\/journal\/(\d+)/';
if(preg_match($regex, $url, $matches) > 0) {
$params['journal-id'] = urldecode($matches[3]);
return $params;
}
// Journals
$regex = '/^(https?:\/\/)?(www\.)?furaffinity.net\/journals\/([^\/&?\n]+)/';
if(preg_match($regex, $url, $matches) > 0) {
$params['username-journals'] = urldecode($matches[3]);
return $params;
}
// Gallery folder
$regex = '/^(https?:\/\/)?(www\.)?furaffinity.net\/gallery\/([^\/&?\n]+)\/folder\/(\d+)/';
if(preg_match($regex, $url, $matches) > 0) {
$params['username-folder'] = urldecode($matches[3]);
$params['folder-id'] = urldecode($matches[4]);
$params['full'] = 'on';
return $params;
}
// Gallery (must be after gallery folder)
$regex = '/^(https?:\/\/)?(www\.)?furaffinity.net\/(gallery|scraps|favorites)\/([^\/&?\n]+)/';
if(preg_match($regex, $url, $matches) > 0) {
$params['username-' . $matches[3]] = urldecode($matches[4]);
$params['full'] = 'on';
return $params;
}
return null;
}
public function getName() {
switch($this->queriedContext) {
case 'Search':
return 'Search For '
. $this->getInput('q');
case 'Browse':
return 'Browse';
case 'Journals':
return $this->getInput('username-journals');
case 'Single Journal':
return 'Journal '
. $this->getInput('journal-id');
case 'Gallery':
return $this->getInput('username-gallery');
case 'Scraps':
return $this->getInput('username-scraps');
case 'Favorites':
return $this->getInput('username-favorites');
case 'Gallery Folder':
return $this->getInput('username-folder')
. '\'s Folder '
. $this->getInput('folder-id');
default: return parent::getName();
}
}
public function getDescription() {
switch($this->queriedContext) {
case 'Search':
return 'FurAffinity Search For '
. $this->getInput('q');
case 'Browse':
return 'FurAffinity Browse';
case 'Journals':
return 'FurAffinity Journals By '
. $this->getInput('username-journals');
case 'Single Journal':
return 'FurAffinity Journal '
. $this->getInput('journal-id');
case 'Gallery':
return 'FurAffinity Gallery By '
. $this->getInput('username-gallery');
case 'Scraps':
return 'FurAffinity Scraps By '
. $this->getInput('username-scraps');
case 'Favorites':
return 'FurAffinity Favorites By '
. $this->getInput('username-favorites');
case 'Gallery Folder':
return 'FurAffinity Gallery Folder '
. $this->getInput('folder-id')
. ' By '
. $this->getInput('username-folder');
default: return parent::getDescription();
}
}
public function getURI() {
switch($this->queriedContext) {
case 'Search':
return SELF::URI
. '/search';
case 'Browse':
return SELF::URI
. '/browse';
case 'Journals':
return SELF::URI
. '/journals/'
. $this->getInput('username-journals');
case 'Single Journal':
return SELF::URI
. '/journal/'
. $this->getInput('journal-id');
case 'Gallery':
return SELF::URI
. '/gallery/'
. $this->getInput('username-gallery');
case 'Scraps':
return SELF::URI
. '/scraps/'
. $this->getInput('username-scraps');
case 'Favorites':
return SELF::URI
. '/favorites/'
. $this->getInput('username-favorites');
case 'Gallery Folder':
return SELF::URI
. '/gallery/'
. $this->getInput('username-folder')
. '/folder/'
. $this->getInput('folder-id');
default: return parent::getURI();
}
}
public function collectData() {
switch($this->queriedContext) {
case 'Search':
$data = array(
'q' => $this->getInput('q'),
'perpage' => 72,
'rating-general' => ($this->getInput('rating-general') === true ? 'on' : 0),
'rating-mature' => ($this->getInput('rating-mature') === true ? 'on' : 0),
'rating-adult' => ($this->getInput('rating-adult') === true ? 'on' : 0),
'range' => $this->getInput('range'),
'type-art' => ($this->getInput('type-art') === true ? 'on' : 0),
'type-flash' => ($this->getInput('type-flash') === true ? 'on' : 0),
'type-photo' => ($this->getInput('type-photo') === true ? 'on' : 0),
'type-music' => ($this->getInput('type-music') === true ? 'on' : 0),
'type-story' => ($this->getInput('type-story') === true ? 'on' : 0),
'type-poetry' => ($this->getInput('type-poetry') === true ? 'on' : 0),
'mode' => $this->getInput('mode')
);
$html = $this->postFASimpleHTMLDOM($data);
$limit = (is_int($this->getInput('limit')) ? $this->getInput('limit') : 10);
$this->itemsFromSubmissionList($html, $limit);
break;
case 'Browse':
$data = array(
'cat' => $this->getInput('cat'),
'atype' => $this->getInput('atype'),
'species' => $this->getInput('species'),
'gender' => $this->getInput('gender'),
'perpage' => 72,
'rating_general' => ($this->getInput('rating_general') === true ? 'on' : 0),
'rating_mature' => ($this->getInput('rating_mature') === true ? 'on' : 0),
'rating_adult' => ($this->getInput('rating_adult') === true ? 'on' : 0)
);
$html = $this->postFASimpleHTMLDOM($data);
$limit = (is_int($this->getInput('limit-browse')) ? $this->getInput('limit-browse') : 10);
$this->itemsFromSubmissionList($html, $limit);
break;
case 'Journals':
$html = $this->getFASimpleHTMLDOM($this->getURI());
$limit = (is_int($this->getInput('limit')) ? $this->getInput('limit') : -1);
$this->itemsFromJournalList($html, $limit);
break;
case 'Single Journal':
$html = $this->getFASimpleHTMLDOM($this->getURI());
$this->itemsFromJournal($html);
break;
case 'Gallery':
case 'Scraps':
case 'Favorites':
case 'Gallery Folder':
$html = $this->getFASimpleHTMLDOM($this->getURI());
$limit = (is_int($this->getInput('limit')) ? $this->getInput('limit') : 10);
$this->itemsFromSubmissionList($html, $limit);
break;
}
}
private function postFASimpleHTMLDOM($data) {
$opts = array(
CURLOPT_CUSTOMREQUEST => 'POST',
CURLOPT_POSTFIELDS => http_build_query($data)
);
$header = array(
'Host: ' . parse_url(self::URI, PHP_URL_HOST),
'Content-Type: application/x-www-form-urlencoded',
'Cookie: ' . self::FA_AUTH_COOKIE
);
$html = getSimpleHTMLDOM($this->getURI(), $header, $opts);
$html = defaultLinkTo($html, $this->getURI());
return $html;
}
private function getFASimpleHTMLDOM($url, $cache = false) {
$header = array(
'Cookie: ' . self::FA_AUTH_COOKIE
);
if($cache) {
$html = getSimpleHTMLDOMCached($url, 86400, $header); // 24 hours
} else {
$html = getSimpleHTMLDOM($url, $header);
}
$html = defaultLinkTo($html, $url);
return $html;
}
private function itemsFromJournalList($html, $limit) {
foreach($html->find('table[id^=jid:]') as $journal) {
# allows limit = -1 to mean 'unlimited'
if($limit-- === 0) break;
$item = array();
$this->setReferrerPolicy($journal);
$item['uri'] = $journal->find('a', 0)->href;
$item['title'] = html_entity_decode($journal->find('a', 0)->plaintext);
$item['author'] = $this->getInput('username-journals');
$item['timestamp'] = strtotime(
$journal->find('span.popup_date', 0)->plaintext);
$item['content'] = $journal
->find('.alt1 table div.no_overflow', 0)
->innertext;
$this->items[] = $item;
}
}
private function itemsFromJournal($html) {
$this->setReferrerPolicy($html);
$item = array();
$item['uri'] = $this->getURI();
$title = $html->find('.journal-title-box .no_overflow', 0)->plaintext;
$title = html_entity_decode($title);
$title = trim($title, " \t\n\r\0\x0B" . chr(0xC2) . chr(0xA0));
$item['title'] = $title;
$item['author'] = $html->find('.journal-title-box a', 0)->plaintext;
$item['timestamp'] = strtotime(
$html->find('.journal-title-box span.popup_date', 0)->plaintext);
$item['content'] = $html->find('.journal-body', 0)->innertext;
$this->items[] = $item;
}
private function itemsFromSubmissionList($html, $limit) {
$cache = ($this->getInput('cache') === true);
foreach($html->find('section.gallery figure') as $figure) {
# allows limit = -1 to mean 'unlimited'
if($limit-- === 0) break;
$item = array();
$submissionURL = $figure->find('b u a', 0)->href;
$imgURL = 'https:' . $figure->find('b u a img', 0)->src;
$item['uri'] = $submissionURL;
$item['title'] = html_entity_decode(
$figure->find('figcaption p a[href*=/view/]', 0)->title);
$item['author'] = $figure->find('figcaption p a[href*=/user/]', 0)->title;
if($this->getInput('full') === true) {
$submissionHTML = $this->getFASimpleHTMLDOM($submissionURL, $cache);
$stats = $submissionHTML->find('.stats-container', 0);
$item['timestamp'] = strtotime($stats->find('.popup_date', 0)->title);
$item['enclosures'] = array(
$submissionHTML->find('.actions a[href^=https://d.facdn]', 0)->href
);
foreach($stats->find('#keywords a') as $keyword) {
$item['categories'][] = $keyword->plaintext;
}
$previewSrc = $submissionHTML->find('#submissionImg', 0)
->{'data-preview-src'};
if($previewSrc) {
$imgURL = 'https:' . $previewSrc;
}
$description = $submissionHTML
->find('.maintable .maintable tr td.alt1', -1);
$this->setReferrerPolicy($description);
$description = $description->innertext;
$item['content'] = <<<EOD
<a href="$submissionURL">
<img src="{$imgURL}" referrerpolicy="no-referrer" />
</a>
<p>
{$description}
</p>
EOD;
} else {
$item['content'] = <<<EOD
<a href="$submissionURL">
<img src="$imgURL" referrerpolicy="no-referrer" />
</a>
EOD;
}
$this->items[] = $item;
}
}
private function setReferrerPolicy(&$html) {
foreach($html->find('img') as $img) {
/*
* Note: Without the no-referrer policy their CDN sometimes denies requests.
* We can't control this for enclosures sadly.
* At least tt-rss adds the referrerpolicy on its own.
* Alternatively we could not use https for images, but that's not ideal.
*/
$img->referrerpolicy = 'no-referrer';
}
}
}

View File

@ -0,0 +1,110 @@
<?php
class FurAffinityUserBridge extends BridgeAbstract {
const NAME = 'FurAffinity User Gallery';
const URI = 'https://www.furaffinity.net';
const MAINTAINER = 'CyberJacob';
const PARAMETERS = array(
array(
'searchUsername' => array(
'name' => 'Search Username',
'type' => 'text',
'required' => true,
'title' => 'Username to fetch the gallery for'
),
'loginUsername' => array(
'name' => 'Login Username',
'type' => 'text',
'required' => true
),
'loginPassword' => array(
'name' => 'Login Password',
'type' => 'text',
'required' => true
)
)
);
public function collectData() {
$cookies = self::login();
$url = self::URI . '/gallery/' . $this->getInput('searchUsername');
$html = getSimpleHTMLDOM($url, $cookies)
or returnServerError('Could not load the user\'s galary page.');
$submissions = $html->find('section[id=gallery-gallery]', 0)->find('figure');
foreach($submissions as $submission) {
$item = array();
$item['title'] = $submission->find('figcaption', 0)->find('a', 0)->plaintext;
$thumbnail = $submission->find('a', 0);
$thumbnail->href = self::URI . $thumbnail->href;
$item['content'] = $submission->find('a', 0);
$this->items[] = $item;
}
}
public function getName() {
return self::NAME . ' for ' . $this->getInput('searchUsername');
}
public function getURI() {
return self::URI . '/user/' . $this->getInput('searchUsername');
}
private function login() {
$ch = curl_init(self::URI . '/login/');
curl_setopt($ch, CURLOPT_RETURNTRANSFER, true);
curl_setopt($ch, CURLOPT_FOLLOWLOCATION, true);
curl_setopt($ch, CURLOPT_USERAGENT, ini_get('user_agent'));
curl_setopt($ch, CURLOPT_ENCODING, '');
curl_setopt($ch, CURLOPT_PROTOCOLS, CURLPROTO_HTTP | CURLPROTO_HTTPS);
$fields = implode('&', array(
'action=login',
'retard_protection=1',
'name=' . urlencode($this->getInput('loginUsername')),
'pass=' . urlencode($this->getInput('loginPassword')),
'login=Login to Faraffinity'
));
curl_setopt($ch, CURLOPT_POST, 5);
curl_setopt($ch, CURLOPT_POSTFIELDS, $fields);
if(defined('PROXY_URL') && !defined('NOPROXY')) {
curl_setopt($ch, CURLOPT_PROXY, PROXY_URL);
}
curl_setopt($ch, CURLOPT_HEADER, true);
curl_setopt($ch, CURLINFO_HEADER_OUT, true);
$data = curl_exec($ch);
$errorCode = curl_getinfo($ch, CURLINFO_HTTP_CODE);
$curlError = curl_error($ch);
$curlErrno = curl_errno($ch);
$curlInfo = curl_getinfo($ch);
if($data === false)
fDebug::log("Cant't download {$url} cUrl error: {$curlError} ({$curlErrno})");
curl_close($ch);
if($errorCode != 200) {
returnServerError(error_get_last());
} else {
preg_match_all('/^Set-Cookie:\s*([^;]*)/mi', $data, $matches);
$cookies = array();
foreach($matches[1] as $item) {
parse_str($item, $cookie);
$cookies = array_merge($cookies, $cookie);
}
return $cookies;
}
}
}

View File

@ -40,6 +40,11 @@ class GQMagazineBridge extends BridgeAbstract
'data-original' => 'src'
);
const POSSIBLE_TITLES = array(
'h2',
'h3'
);
private function getDomain() {
$domain = $this->getInput('domain');
if (empty($domain))
@ -54,6 +59,17 @@ class GQMagazineBridge extends BridgeAbstract
return $this->getDomain() . '/' . $this->getInput('page');
}
private function findTitleOf($link) {
foreach (self::POSSIBLE_TITLES as $tag) {
$title = $link->parent()->find($tag, 0);
if($title !== null) {
if($title->plaintext !== null) {
return $title->plaintext;
}
}
}
}
public function collectData()
{
$html = getSimpleHTMLDOM($this->getURI()) or returnServerError('Could not request ' . $this->getURI());
@ -61,31 +77,36 @@ class GQMagazineBridge extends BridgeAbstract
// Since GQ don't want simple class scrapping, let's do it the hard way and ... discover content !
$main = $html->find('main', 0);
foreach ($main->find('a') as $link) {
if(strpos($link, $this->getInput('page')))
continue;
$uri = $link->href;
$title = $link->find('h2', 0);
$date = $link->find('time', 0);
$date = $link->parent()->find('time', 0);
$item = array();
$author = $link->find('span[itemprop=name]', 0);
$item['author'] = $author->plaintext;
$item['title'] = $title->plaintext;
if(substr($uri, 0, 1) === 'h') { // absolute uri
$item['uri'] = $uri;
} else if(substr($uri, 0, 1) === '/') { // domain relative url
$item['uri'] = $this->getDomain() . $uri;
} else {
$item['uri'] = $this->getDomain() . '/' . $uri;
$author = $link->parent()->find('span[itemprop=name]', 0);
if($author !== null) {
$item['author'] = $author->plaintext;
$item['title'] = $this->findTitleOf($link);
switch(substr($uri, 0, 1)) {
case 'h': // absolute uri
$item['uri'] = $uri;
break;
case '/': // domain relative uri
$item['uri'] = $this->getDomain() . $uri;
break;
default:
$item['uri'] = $this->getDomain() . '/' . $uri;
}
$article = $this->loadFullArticle($item['uri']);
if($article) {
$item['content'] = $this->replaceUriInHtmlElement($article);
} else {
$item['content'] = "<strong>Article body couldn't be loaded</strong>. It must be a bug!";
}
$short_date = $date->datetime;
$item['timestamp'] = strtotime($short_date);
$this->items[] = $item;
}
$article = $this->loadFullArticle($item['uri']);
if($article) {
$item['content'] = $this->replaceUriInHtmlElement($article);
} else {
$item['content'] = "<strong>Article body couldn't be loaded</strong>. It must be a bug!";
}
$short_date = $date->datetime;
$item['timestamp'] = strtotime($short_date);
$this->items[] = $item;
}
}
@ -96,16 +117,7 @@ class GQMagazineBridge extends BridgeAbstract
*/
private function loadFullArticle($uri){
$html = getSimpleHTMLDOMCached($uri);
// Once again, that generated css classes madness is an obstacle ... which i can go over easily
foreach($html->find('div') as $div) {
// List the CSS classes of that div
$classes = $div->class;
// I can't directly lookup that class since GQ since to generate random names like "ArticleBodySection-fkggUW"
if(strpos($classes, 'ArticleBodySection') !== false) {
return $div;
}
}
return null;
return $html->find('section[data-test-id=MainContentWrapper]', 0);
}
/**

View File

@ -5,7 +5,7 @@ class GiphyBridge extends BridgeAbstract {
const MAINTAINER = 'kraoc';
const NAME = 'Giphy Bridge';
const URI = 'http://giphy.com/';
const URI = 'https://giphy.com/';
const CACHE_TIMEOUT = 300; //5min
const DESCRIPTION = 'Bridge for giphy.com';

27
bridges/GiteaBridge.php Normal file
View File

@ -0,0 +1,27 @@
<?php
/**
* Gitea is a fork of Gogs which may diverge in the future.
* https://docs.gitea.io/en-us/
*/
require_once 'GogsBridge.php';
class GiteaBridge extends GogsBridge {
const NAME = 'Gitea';
const URI = 'https://gitea.io';
const DESCRIPTION = 'Returns the latest issues, commits or releases';
const MAINTAINER = 'logmanoriginal';
const CACHE_TIMEOUT = 300; // 5 minutes
protected function collectReleasesData($html) {
$releases = $html->find('#release-list > li')
or returnServerError('Unable to find releases');
foreach($releases as $release) {
$this->items[] = array(
'uri' => $release->find('a', 0)->href,
'title' => 'Release ' . $release->find('h3', 0)->plaintext,
);
}
}
}

View File

@ -66,10 +66,21 @@ class GithubIssueBridge extends BridgeAbstract {
return parent::getURI();
}
protected function extractIssueEvent($issueNbr, $title, $comment){
$comment = $comment->firstChild();
$uri = static::URI . $this->getInput('u') . '/' . $this->getInput('p')
. '/issues/' . $issueNbr . '#' . $comment->getAttribute('id');
private function buildGitHubIssueCommentUri($issue_number, $comment_id) {
// https://github.com/<user>/<project>/issues/<issue-number>#<id>
return static::URI
. $this->getInput('u')
. '/'
. $this->getInput('p')
. '/issues/'
. $issue_number
. '#'
. $comment_id;
}
private function extractIssueEvent($issueNbr, $title, $comment){
$uri = $this->buildGitHubIssueCommentUri($issueNbr, $comment->id);
$author = $comment->find('.author', 0)->plaintext;
@ -94,22 +105,21 @@ class GithubIssueBridge extends BridgeAbstract {
return $item;
}
protected function extractIssueComment($issueNbr, $title, $comment){
$uri = static::URI . $this->getInput('u') . '/'
. $this->getInput('p') . '/issues/' . $issueNbr;
private function extractIssueComment($issueNbr, $title, $comment){
$uri = $this->buildGitHubIssueCommentUri($issueNbr, $comment->parent->id);
$author = $comment->find('.author', 0)->plaintext;
$title .= ' / ' . trim(
$comment->find('.comment .timeline-comment-header-text', 0)->plaintext
$comment->find('.timeline-comment-header-text', 0)->plaintext
);
$content = $comment->find('.comment-body', 0)->innertext;
$item = array();
$item['author'] = $author;
$item['uri'] = $uri
. '#' . $comment->firstChild()->nextSibling()->getAttribute('id');
$item['uri'] = $uri;
$item['title'] = html_entity_decode($title, ENT_QUOTES, 'UTF-8');
$item['timestamp'] = strtotime(
$comment->find('relative-time', 0)->getAttribute('datetime')
@ -118,25 +128,32 @@ class GithubIssueBridge extends BridgeAbstract {
return $item;
}
protected function extractIssueComments($issue){
private function extractIssueComments($issue){
$items = array();
$title = $issue->find('.gh-header-title', 0)->plaintext;
$issueNbr = trim(
substr($issue->find('.gh-header-number', 0)->plaintext, 1)
);
$comments = $issue->find('.js-discussion', 0);
foreach($comments->children() as $comment) {
$comments = $issue->find('
[id^="issue-"] > .comment,
[id^="issuecomment-"] > .comment,
[id^="event-"],
[id^="ref-"]
');
foreach($comments as $comment) {
if (!$comment->hasChildNodes()) {
continue;
}
$comment = $comment->firstChild();
$classes = explode(' ', $comment->getAttribute('class'));
if (in_array('timeline-comment-wrapper', $classes)) {
if (!$comment->hasClass('discussion-item-header')) {
$item = $this->extractIssueComment($issueNbr, $title, $comment);
$items[] = $item;
continue;
}
while (in_array('discussion-item', $classes)) {
while ($comment->hasClass('discussion-item-header')) {
$item = $this->extractIssueEvent($issueNbr, $title, $comment);
$items[] = $item;
$comment = $comment->nextSibling();
@ -145,6 +162,7 @@ class GithubIssueBridge extends BridgeAbstract {
}
$classes = explode(' ', $comment->getAttribute('class'));
}
}
return $items;
}
@ -192,8 +210,13 @@ class GithubIssueBridge extends BridgeAbstract {
ENT_QUOTES,
'UTF-8'
);
$comments = trim($issue->find('.col-5', 0)->plaintext);
$item['content'] .= "\n" . 'Comments: ' . ($comments ? $comments : '0');
$comment_count = 0;
if($span = $issue->find('a[aria-label*="comment"] span', 0)) {
$comment_count = $span->plaintext;
}
$item['content'] .= "\n" . 'Comments: ' . $comment_count;
$item['uri'] = self::URI
. $issue->find('.js-navigation-open', 0)->getAttribute('href');
$this->items[] = $item;
@ -216,4 +239,43 @@ class GithubIssueBridge extends BridgeAbstract {
$item['title'] = preg_replace('/\s+/', ' ', $item['title']);
});
}
public function detectParameters($url) {
if(filter_var($url, FILTER_VALIDATE_URL, FILTER_FLAG_PATH_REQUIRED) === false
|| strpos($url, self::URI) !== 0) {
return null;
}
$url_components = parse_url($url);
$path_segments = array_values(array_filter(explode('/', $url_components['path'])));
switch(count($path_segments)) {
case 2: { // Project issues
list($user, $project) = $path_segments;
$show_comments = 'off';
} break;
case 3: { // Project issues with issue comments
if($path_segments[2] !== 'issues') {
return null;
}
list($user, $project) = $path_segments;
$show_comments = 'on';
} break;
case 4: { // Issue comments
list($user, $project, /* issues */, $issue) = $path_segments;
} break;
default: {
return null;
}
}
return array(
'u' => $user,
'p' => $project,
'c' => isset($show_comments) ? $show_comments : null,
'i' => isset($issue) ? $issue : null,
);
}
}

View File

@ -141,7 +141,7 @@ class GlassdoorBridge extends BridgeAbstract {
}
private function collectReviewData($html, $limit) {
$reviews = $html->find('#EmployerReviews li[id^="empReview]')
$reviews = $html->find('#ReviewsFeed li[id^="empReview]')
or returnServerError('Unable to find reviews!');
foreach($reviews as $review) {
@ -153,7 +153,19 @@ class GlassdoorBridge extends BridgeAbstract {
$item['timestamp'] = strtotime($review->find('time', 0)->datetime);
$mainText = $review->find('p.mainText', 0)->plaintext;
$description = $review->find('div.prosConsAdvice', 0)->innertext;
$description = '';
foreach($review->find('div.description p') as $p) {
if ($p->hasClass('strong')) {
$p->tag = 'strong';
$p->removeClass('strong');
}
$description .= $p;
}
$item['content'] = "<p>{$mainText}</p><p>{$description}</p>";
$this->items[] = $item;

206
bridges/GogsBridge.php Normal file
View File

@ -0,0 +1,206 @@
<?php
class GogsBridge extends BridgeAbstract {
const NAME = 'Gogs';
const URI = 'https://gogs.io';
const DESCRIPTION = 'Returns the latest issues, commits or releases';
const MAINTAINER = 'logmanoriginal';
const CACHE_TIMEOUT = 300; // 5 minutes
const PARAMETERS = array(
'global' => array(
'host' => array(
'name' => 'Host',
'exampleValue' => 'https://gogs.io',
'required' => true,
'title' => 'Host name without trailing slash',
),
'user' => array(
'name' => 'Username',
'exampleValue' => 'gogs',
'required' => true,
'title' => 'User name as it appears in the URL',
),
'project' => array(
'name' => 'Project name',
'exampleValue' => 'gogs',
'required' => true,
'title' => 'Project name as it appears in the URL',
),
),
'Commits' => array(
'branch' => array(
'name' => 'Branch name',
'defaultValue' => 'master',
'required' => true,
'title' => 'Branch name as it appears in the URL',
),
),
'Issues' => array(
'include_description' => array(
'name' => 'Include issue description',
'type' => 'checkbox',
'title' => 'Activate to include the issue description',
),
),
'Single issue' => array(
'issue' => array(
'name' => 'Issue number',
'type' => 'number',
'exampleValue' => 102,
'required' => true,
'title' => 'Issue number from the issues list',
),
),
'Releases' => array(),
);
private $title = '';
/**
* Note: detectParamters doesn't make sense for this bridge because there is
* no "single" host for this service. Anyone can host it.
*/
public function getURI() {
switch($this->queriedContext) {
case 'Commits': {
return $this->getInput('host')
. '/' . $this->getInput('user')
. '/' . $this->getInput('project')
. '/commits/' . $this->getInput('branch');
} break;
case 'Issues': {
return $this->getInput('host')
. '/' . $this->getInput('user')
. '/' . $this->getInput('project')
. '/issues/';
} break;
case 'Single issue': {
return $this->getInput('host')
. '/' . $this->getInput('user')
. '/' . $this->getInput('project')
. '/issues/' . $this->getInput('issue');
} break;
case 'Releases': {
return $this->getInput('host')
. '/' . $this->getInput('user')
. '/' . $this->getInput('project')
. '/releases/';
} break;
default: return parent::getURI();
}
}
public function getName() {
switch($this->queriedContext) {
case 'Commits':
case 'Issues':
case 'Releases': return $this->title . ' ' . $this->queriedContext;
case 'Single issue': return $this->title . ' Issue ' . $this->getInput('issue');
default: return parent::getName();
}
}
public function getIcon() {
return 'https://gogs.io/img/favicon.ico';
}
public function collectData() {
$html = getSimpleHTMLDOM($this->getURI())
or returnServerError('Could not request ' . $this->getURI());
$html = defaultLinkTo($html, $this->getURI());
$this->title = $html->find('[property="og:title"]', 0)->content;
switch($this->queriedContext) {
case 'Commits': {
$this->collectCommitsData($html);
} break;
case 'Issues': {
$this->collectIssuesData($html);
} break;
case 'Single issue': {
$this->collectSingleIssueData($html);
} break;
case 'Releases': {
$this->collectReleasesData($html);
} break;
}
}
protected function collectCommitsData($html) {
$commits = $html->find('#commits-table tbody tr')
or returnServerError('Unable to find commits');
foreach($commits as $commit) {
$this->items[] = array(
'uri' => $commit->find('a.sha', 0)->href,
'title' => $commit->find('.message span', 0)->plaintext,
'author' => $commit->find('.author', 0)->plaintext,
'timestamp' => $commit->find('.time-since', 0)->title,
'uid' => $commit->find('.sha', 0)->plaintext,
);
}
}
protected function collectIssuesData($html) {
$issues = $html->find('.issue.list li')
or returnServerError('Unable to find issues');
foreach($issues as $issue) {
$uri = $issue->find('a', 0)->href;
$item = array(
'uri' => $uri,
'title' => $issue->find('.label', 0)->plaintext . ' | ' . $issue->find('a.title', 0)->plaintext,
'author' => $issue->find('.desc a', 0)->plaintext,
'timestamp' => $issue->find('.time-since', 0)->title,
'uid' => $issue->find('.label', 0)->plaintext,
);
if($this->getInput('include_description')) {
$issue_html = getSimpleHTMLDOMCached($uri, 3600)
or returnServerError('Unable to load issue description');
$issue_html = defaultLinkTo($issue_html, $uri);
$item['content'] = $issue_html->find('.comment .markdown', 0);
}
$this->items[] = $item;
}
}
protected function collectSingleIssueData($html) {
$comments = $html->find('.comments .comment')
or returnServerError('Unable to find comments');
foreach($comments as $comment) {
$this->items[] = array(
'uri' => $comment->find('a[href*="#issue"]', 0)->href,
'title' => $comment->find('span', 0)->plaintext,
'author' => $comment->find('.content a', 0)->plaintext,
'timestamp' => $comment->find('.time-since', 0)->title,
'content' => $comment->find('.markdown', 0),
);
}
$this->items = array_reverse($this->items);
}
protected function collectReleasesData($html) {
$releases = $html->find('#release-list li')
or returnServerError('Unable to find releases');
foreach($releases as $release) {
$this->items[] = array(
'uri' => $release->find('a', 0)->href,
'title' => 'Release ' . $release->find('h4', 0)->plaintext,
);
}
}
}

View File

@ -25,13 +25,10 @@ class GoogleSearchBridge extends BridgeAbstract {
public function collectData(){
$html = '';
$html = getSimpleHTMLDOM(self::URI
. 'search?q='
. urlencode($this->getInput('q'))
. '&num=100&complete=0&tbs=qdr:y,sbd:1')
$html = getSimpleHTMLDOM($this->getURI())
or returnServerError('No results for this query.');
$emIsRes = $html->find('div[id=ires]', 0);
$emIsRes = $html->find('div[id=res]', 0);
if(!is_null($emIsRes)) {
foreach($emIsRes->find('div[class=g]') as $element) {
@ -54,6 +51,17 @@ class GoogleSearchBridge extends BridgeAbstract {
}
}
public function getURI() {
if (!is_null($this->getInput('q'))) {
return self::URI
. 'search?q='
. urlencode($this->getInput('q'))
. '&num=100&complete=0&tbs=qdr:y,sbd:1';
}
return parent::getURI();
}
public function getName(){
if(!is_null($this->getInput('q'))) {
return $this->getInput('q') . ' - Google search';

View File

@ -2,7 +2,7 @@
class HDWallpapersBridge extends BridgeAbstract {
const MAINTAINER = 'nel50n';
const NAME = 'HD Wallpapers Bridge';
const URI = 'http://www.hdwallpapers.in/';
const URI = 'https://www.hdwallpapers.in/';
const CACHE_TIMEOUT = 43200; //12h
const DESCRIPTION = 'Returns the latests wallpapers from HDWallpapers';
@ -72,7 +72,7 @@ class HDWallpapersBridge extends BridgeAbstract {
public function getName(){
if(!is_null($this->getInput('c')) && !is_null($this->getInput('r'))) {
return 'HDWallpapers - '
. str_replace(['__', '_'], [' & ', ' '], $this->getInput('c'))
. str_replace(array('__', '_'), array(' & ', ' '), $this->getInput('c'))
. ' ['
. $this->getInput('r')
. ']';

View File

@ -13,6 +13,11 @@ class HaveIBeenPwnedBridge extends BridgeAbstract {
'Date added to HIBP' => 'dateAdded',
),
'defaultValue' => 'dateAdded',
),
'item_limit' => array(
'name' => 'Limit number of returned items',
'type' => 'number',
'defaultValue' => 20,
)
));
@ -52,13 +57,15 @@ class HaveIBeenPwnedBridge extends BridgeAbstract {
// Remove permalink
$breach->find('p', 1)->find('a', 0)->outertext = '';
$item['title'] = $breach->find('h3', 0)->plaintext . ' - ' . $accounts[1] . ' breached accounts';
$item['title'] = html_entity_decode($breach->find('h3', 0)->plaintext, ENT_QUOTES)
. ' - ' . $accounts[1] . ' breached accounts';
$item['dateAdded'] = strtotime($dateAdded[1]);
$item['breachDate'] = strtotime($breachDate[1]);
$item['uri'] = self::URI . '/PwnedWebsites' . $permalink;
$item['content'] = '<p>' . $breach->find('p', 0)->innertext . '<p>';
$item['content'] .= '<p>' . $breach->find('p', 1)->innertext . '<p>';
$item['content'] = '<p>' . $breach->find('p', 0)->innertext . '</p>';
$item['content'] .= '<p>' . $this->breachType($breach) . '</p>';
$item['content'] .= '<p>' . $breach->find('p', 1)->innertext . '</p>';
$this->breaches[] = $item;
}
@ -67,6 +74,25 @@ class HaveIBeenPwnedBridge extends BridgeAbstract {
$this->createItems();
}
/**
* Extract data breach type(s)
*/
private function breachType($breach) {
$content = '';
if ($breach->find('h3 > i', 0)) {
foreach ($breach->find('h3 > i') as $i) {
$content .= $i->title . '.<br>';
}
}
return $content;
}
/**
* Order Breaches by date added or date breached
*/
@ -88,6 +114,12 @@ class HaveIBeenPwnedBridge extends BridgeAbstract {
*/
private function createItems() {
$limit = $this->getInput('item_limit');
if ($limit < 1) {
$limit = 20;
}
foreach ($this->breaches as $breach) {
$item = array();
@ -97,6 +129,10 @@ class HaveIBeenPwnedBridge extends BridgeAbstract {
$item['content'] = $breach['content'];
$this->items[] = $item;
if (count($this->items) >= $limit) {
break;
}
}
}
}

View File

@ -3,7 +3,7 @@ class HentaiHavenBridge extends BridgeAbstract {
const MAINTAINER = 'albirew';
const NAME = 'Hentai Haven';
const URI = 'http://hentaihaven.org/';
const URI = 'https://hentaihaven.org/';
const CACHE_TIMEOUT = 21600; // 6h
const DESCRIPTION = 'Returns releases from Hentai Haven';

55
bridges/IGNBridge.php Normal file
View File

@ -0,0 +1,55 @@
<?php
class IGNBridge extends FeedExpander {
const MAINTAINER = 'IceWreck';
const NAME = 'IGN Bridge';
const URI = 'https://www.ign.com/';
const CACHE_TIMEOUT = 3600;
const DESCRIPTION = 'RSS Feed For IGN';
public function collectData(){
$this->collectExpandableDatas('http://feeds.ign.com/ign/all', 15);
}
// IGNs feed is both hidden and incomplete. This bridge tries to fix this.
protected function parseItem($newsItem){
$item = parent::parseItem($newsItem);
// $articlePage gets the entire page's contents
$articlePage = getSimpleHTMLDOM($newsItem->link);
/*
* NOTE: Though articles and wiki/howtos have seperate styles of pages, there is no mechanism
* for handling them seperately as it just ignores the DOM querys which it does not find.
* (and their scraping)
*/
// For Articles
$article = $articlePage->find('section.article-page', 0);
// add in verdicts in articles, reviews etc
foreach($articlePage->find('div.article-section') as $element) {
$article = $article . $element;
}
// For Wikis and HowTos
$uselessWikiElements = array(
'.wiki-page-tools',
'.feedback-container',
'.paging-container'
);
foreach($articlePage->find('.wiki-page') as $wikiContents) {
$copy = clone $wikiContents;
// Remove useless elements present in IGN wiki/howtos
foreach($uselessWikiElements as $uslElement) {
$toRemove = $wikiContents->find($uslElement, 0);
$copy = str_replace($toRemove, '', $copy);
}
$article = $article . $copy;
}
// Add content to feed
$item['content'] = $article;
return $item;
}
}

245
bridges/IndeedBridge.php Normal file
View File

@ -0,0 +1,245 @@
<?php
class IndeedBridge extends BridgeAbstract {
const NAME = 'Indeed';
const URI = 'https://www.indeed.com/';
const DESCRIPTION = 'Returns reviews and comments for a company of your choice';
const MAINTAINER = 'logmanoriginal';
const CACHE_TIMEOUT = 14400; // 4 hours
const PARAMETERS = array(
array(
'c' => array(
'name' => 'Company',
'type' => 'text',
'required' => true,
'title' => 'Company name',
'exampleValue' => 'GitHub',
)
),
'global' => array(
'language' => array(
'name' => 'Language Code',
'type' => 'list',
'title' => 'Choose your language code',
'defaultValue' => 'en-US',
'values' => array(
'es-AR' => 'es-AR',
'de-AT' => 'de-AT',
'en-AU' => 'en-AU',
'nl-BE' => 'nl-BE',
'fr-BE' => 'fr-BE',
'pt-BR' => 'pt-BR',
'en-CA' => 'en-CA',
'fr-CA' => 'fr-CA',
'de-CH' => 'de-CH',
'fr-CH' => 'fr-CH',
'es-CL' => 'es-CL',
'zh-CN' => 'zh-CN',
'es-CO' => 'es-CO',
'de-DE' => 'de-DE',
'es-ES' => 'es-ES',
'fr-FR' => 'fr-FR',
'en-GB' => 'en-GB',
'en-HK' => 'en-HK',
'en-IE' => 'en-IE',
'en-IN' => 'en-IN',
'it-IT' => 'it-IT',
'ja-JP' => 'ja-JP',
'ko-KR' => 'ko-KR',
'es-MX' => 'es-MX',
'nl-NL' => 'nl-NL',
'pl-PL' => 'pl-PL',
'en-SG' => 'en-SG',
'en-US' => 'en-US',
'en-ZA' => 'en-ZA',
'en-AE' => 'en-AE',
'da-DK' => 'da-DK',
'in-ID' => 'in-ID',
'en-MY' => 'en-MY',
'es-PE' => 'es-PE',
'en-PH' => 'en-PH',
'en-PK' => 'en-PK',
'ro-RO' => 'ro-RO',
'ru-RU' => 'ru-RU',
'tr-TR' => 'tr-TR',
'zh-TW' => 'zh-TW',
'vi-VN' => 'vi-VN',
'en-VN' => 'en-VN',
'ar-EG' => 'ar-EG',
'fr-MA' => 'fr-MA',
'en-NG' => 'en-NG',
)
),
'limit' => array(
'name' => 'Limit',
'type' => 'number',
'title' => 'Maximum number of items to return',
'exampleValue' => 20,
)
)
);
const SITES = array(
'es-AR' => 'https://ar.indeed.com/',
'de-AT' => 'https://at.indeed.com/',
'en-AU' => 'https://au.indeed.com/',
'nl-BE' => 'https://be.indeed.com/',
'fr-BE' => 'https://emplois.be.indeed.com/',
'pt-BR' => 'https://www.indeed.com.br/',
'en-CA' => 'https://ca.indeed.com/',
'fr-CA' => 'https://emplois.ca.indeed.com/',
'de-CH' => 'https://www.indeed.ch/',
'fr-CH' => 'https://emplois.indeed.ch/',
'es-CL' => 'https://www.indeed.cl/',
'zh-CN' => 'https://cn.indeed.com/',
'es-CO' => 'https://co.indeed.com/',
'de-DE' => 'https://de.indeed.com/',
'es-ES' => 'https://www.indeed.es/',
'fr-FR' => 'https://www.indeed.fr/',
'en-GB' => 'https://www.indeed.co.uk/',
'en-HK' => 'https://www.indeed.hk/',
'en-IE' => 'https://ie.indeed.com/',
'en-IN' => 'https://www.indeed.co.in/',
'it-IT' => 'https://it.indeed.com/',
'ja-JP' => 'https://jp.indeed.com/',
'ko-KR' => 'https://kr.indeed.com/',
'es-MX' => 'https://www.indeed.com.mx/',
'nl-NL' => 'https://www.indeed.nl/',
'pl-PL' => 'https://pl.indeed.com/',
'en-SG' => 'https://www.indeed.com.sg/',
'en-US' => 'https://www.indeed.com/',
'en-ZA' => 'https://www.indeed.co.za/',
'en-AE' => 'https://www.indeed.ae/',
'da-DK' => 'https://dk.indeed.com/',
'in-ID' => 'https://id.indeed.com/',
'en-MY' => 'https://www.indeed.com.my/',
'es-PE' => 'https://www.indeed.com.pe/',
'en-PH' => 'https://www.indeed.com.ph/',
'en-PK' => 'https://www.indeed.com.pk/',
'ro-RO' => 'https://ro.indeed.com/',
'ru-RU' => 'https://ru.indeed.com/',
'tr-TR' => 'https://tr.indeed.com/',
'zh-TW' => 'https://tw.indeed.com/',
'vi-VN' => 'https://vn.indeed.com/',
'en-VN' => 'https://jobs.vn.indeed.com/',
'ar-EG' => 'https://eg.indeed.com/',
'fr-MA' => 'https://ma.indeed.com/',
'en-NG' => 'https://ng.indeed.com/',
);
private $title;
public function collectData() {
$url = $this->getURI();
$limit = $this->getInput('limit') ?: 20;
do {
$html = getSimpleHTMLDOM($url)
or returnServerError('Could not request ' . $url);
$html = defaultLinkTo($html, $url);
$this->title = $html->find('h1', 0)->innertext;
// Use local translation of the word "Rating"
$rating_local = $html->find('a[data-id="rating_desc"]', 0)->plaintext;
foreach($html->find('#cmp-content [id^="cmp-review-"]') as $review) {
$item = array();
$rating = $review->find('.cmp-ratingNumber', 0)->plaintext;
$title = $review->find('.cmp-review-title > span', 0)->plaintext;
$comment = $this->beautifyComment($review->find('.cmp-review-content-container', 0));
$item['uri'] = $review->find('.cmp-review-share-popup-item-link--copylink', 0)->href;
$item['title'] = "{$rating_local} {$rating} / {$title}";
$item['timestamp'] = $review->find('.cmp-review-date-created', 0)->plaintext;
$item['author'] = $review->find('.cmp-reviewer', 0)->plaintext;
$item['content'] = $comment;
//$item['enclosures']
$item['categories'][] = $review->find('.cmp-reviewer-job-location', 0)->plaintext;
//$item['uid']
$this->items[] = $item;
if(count($this->items) >= $limit) {
break;
}
}
// Break if no more pages available.
if($next = $html->find('a[data-tn-element="next-page"]', 0)) {
$url = $next->href;
} else {
break;
}
} while(count($this->items) < $limit);
}
public function getURI() {
if($this->getInput('language')
&& $this->getInput('c')) {
return self::SITES[$this->getInput('language')]
. 'cmp/'
. urlencode($this->getInput('c'))
. '/reviews';
}
return parent::getURI();
}
public function getName() {
return $this->title ?: parent::getName();
}
public function detectParameters($url) {
/**
* Expected: https://<...>.indeed.<...>/cmp/<company>[/reviews][/...]
*
* Note that most users will be redirected to their localized version
* of the page, which adds the language code to the host. For example,
* "en.indeed.com" or "www.indeed.fr" (see link[rel="alternate"]). At
* least each of the sites have ".indeed." in the name.
*/
if(filter_var($url, FILTER_VALIDATE_URL, FILTER_FLAG_PATH_REQUIRED) === false
|| stristr($url, '.indeed.') === false) {
return null;
}
$url_components = parse_url($url);
$path_segments = array_values(array_filter(explode('/', $url_components['path'])));
if(count($path_segments) < 2 || $path_segments[0] !== 'cmp') {
return null;
}
$language = array_search('https://' . $url_components['host'] . '/', self::SITES);
if($language === false) {
return null;
}
$limit = self::PARAMETERS['global']['limit']['defaultValue'] ?: 20;
$company = $path_segments[1];
return array(
'c' => $company,
'language' => $language,
'limit' => $limit,
);
}
private function beautifyComment($comment) {
foreach($comment->find('.cmp-bold') as $bold) {
$bold->tag = 'strong';
$bold->removeClass('cmp-bold');
}
return $comment;
}
}

View File

@ -32,28 +32,61 @@ class InstagramBridge extends BridgeAbstract {
'required' => false,
'values' => array(
'All' => 'all',
'Story' => 'story',
'Video' => 'video',
'Picture' => 'picture',
'Multiple' => 'multiple',
),
'defaultValue' => 'all'
),
'direct_links' => array(
'name' => 'Use direct media links',
'type' => 'checkbox',
)
)
);
public function collectData(){
const USER_QUERY_HASH = '58b6785bea111c67129decbe6a448951';
const TAG_QUERY_HASH = '174a5243287c5f3a7de741089750ab3b';
const SHORTCODE_QUERY_HASH = '865589822932d1b43dfe312121dd353a';
if(is_null($this->getInput('u')) && $this->getInput('media_type') == 'story') {
returnClientError('Stories are not supported for hashtags nor locations!');
protected function getInstagramUserId($username) {
if(is_numeric($username)) return $username;
$cacheFac = new CacheFactory();
$cacheFac->setWorkingDir(PATH_LIB_CACHES);
$cache = $cacheFac->create(Configuration::getConfig('cache', 'type'));
$cache->setScope(get_called_class());
$cache->setKey(array($username));
$key = $cache->loadData();
if($key == null) {
$data = getContents(self::URI . 'web/search/topsearch/?query=' . $username);
foreach(json_decode($data)->users as $user) {
if($user->user->username === $username) {
$key = $user->user->pk;
}
}
if($key == null) {
returnServerError('Unable to find username in search result.');
}
$cache->saveData($key);
}
return $key;
}
public function collectData(){
$directLink = !is_null($this->getInput('direct_links')) && $this->getInput('direct_links');
$data = $this->getInstagramJSON($this->getURI());
if(!is_null($this->getInput('u'))) {
$userMedia = $data->entry_data->ProfilePage[0]->graphql->user->edge_owner_to_timeline_media->edges;
$userMedia = $data->data->user->edge_owner_to_timeline_media->edges;
} elseif(!is_null($this->getInput('h'))) {
$userMedia = $data->entry_data->TagPage[0]->graphql->hashtag->edge_hashtag_to_media->edges;
$userMedia = $data->data->hashtag->edge_hashtag_to_media->edges;
} elseif(!is_null($this->getInput('l'))) {
$userMedia = $data->entry_data->LocationsPage[0]->graphql->location->edge_location_to_media->edges;
}
@ -61,22 +94,18 @@ class InstagramBridge extends BridgeAbstract {
foreach($userMedia as $media) {
$media = $media->node;
if(!is_null($this->getInput('u'))) {
switch($this->getInput('media_type')) {
case 'all': break;
case 'video':
if($media->__typename != 'GraphVideo') continue 2;
break;
case 'picture':
if($media->__typename != 'GraphImage') continue 2;
break;
case 'story':
if($media->__typename != 'GraphSidecar') continue 2;
break;
default: break;
}
} else {
if($this->getInput('media_type') == 'video' && !$media->is_video) continue;
switch($this->getInput('media_type')) {
case 'all': break;
case 'video':
if($media->__typename != 'GraphVideo' || !$media->is_video) continue 2;
break;
case 'picture':
if($media->__typename != 'GraphImage') continue 2;
break;
case 'multiple':
if($media->__typename != 'GraphSidecar') continue 2;
break;
default: break;
}
$item = array();
@ -86,72 +115,141 @@ class InstagramBridge extends BridgeAbstract {
$item['author'] = $media->owner->username;
}
if (isset($media->edge_media_to_caption->edges[0]->node->text)) {
$textContent = $media->edge_media_to_caption->edges[0]->node->text;
} else {
$textContent = '(no text)';
}
$textContent = $this->getTextContent($media);
$item['title'] = ($media->is_video ? '▶ ' : '') . trim($textContent);
$item['title'] = ($media->is_video ? '▶ ' : '') . $textContent;
$titleLinePos = strpos(wordwrap($item['title'], 120), "\n");
if ($titleLinePos != false) {
$item['title'] = substr($item['title'], 0, $titleLinePos) . '...';
}
if(!is_null($this->getInput('u')) && $media->__typename == 'GraphSidecar') {
$data = $this->getInstagramStory($item['uri']);
$item['content'] = $data[0];
$item['enclosures'] = $data[1];
} else {
$mediaURI = self::URI . 'p/' . $media->shortcode . '/media?size=l';
$item['content'] = '<a href="' . htmlentities($item['uri']) . '" target="_blank">';
$item['content'] .= '<img src="' . htmlentities($mediaURI) . '" alt="' . $item['title'] . '" />';
$item['content'] .= '</a><br><br>' . nl2br(htmlentities($textContent));
$item['enclosures'] = array($mediaURI);
switch($media->__typename) {
case 'GraphSidecar':
$data = $this->getInstagramSidecarData($item['uri'], $item['title']);
$item['content'] = $data[0];
$item['enclosures'] = $data[1];
break;
case 'GraphImage':
if($directLink) {
$mediaURI = $media->display_url;
} else {
$mediaURI = self::URI . 'p/' . $media->shortcode . '/media?size=l';
}
$item['content'] = '<a href="' . htmlentities($item['uri']) . '" target="_blank">';
$item['content'] .= '<img src="' . htmlentities($mediaURI) . '" alt="' . $item['title'] . '" />';
$item['content'] .= '</a><br><br>' . nl2br(htmlentities($textContent));
$item['enclosures'] = array($mediaURI);
break;
case 'GraphVideo':
$data = $this->getInstagramVideoData($item['uri']);
$item['content'] = $data[0];
if($directLink) {
$item['enclosures'] = $data[1];
} else {
$item['enclosures'] = array(self::URI . 'p/' . $media->shortcode . '/media?size=l');
}
break;
default: break;
}
$item['timestamp'] = $media->taken_at_timestamp;
$this->items[] = $item;
}
}
protected function getInstagramStory($uri) {
// returns Sidecar(a post which has multiple media)'s contents and enclosures
protected function getInstagramSidecarData($uri, $postTitle) {
$mediaInfo = $this->getSinglePostData($uri);
$data = $this->getInstagramJSON($uri);
$mediaInfo = $data->entry_data->PostPage[0]->graphql->shortcode_media;
$textContent = $this->getTextContent($mediaInfo);
//Process the first element, that isn't in the node graph
if (count($mediaInfo->edge_media_to_caption->edges) > 0) {
$caption = $mediaInfo->edge_media_to_caption->edges[0]->node->text;
} else {
$caption = '';
}
$enclosures = [$mediaInfo->display_url];
$content = '<img src="' . htmlentities($mediaInfo->display_url) . '" alt="' . $caption . '" />';
foreach($mediaInfo->edge_sidecar_to_children->edges as $media) {
$display_url = $media->node->display_url;
if(!in_array($display_url, $enclosures)) { // add only if not added yet
$content .= '<img src="' . htmlentities($display_url) . '" alt="' . $caption . '" />';
$enclosures[] = $display_url;
$enclosures = array();
$content = '';
foreach($mediaInfo->edge_sidecar_to_children->edges as $singleMedia) {
$singleMedia = $singleMedia->node;
if($singleMedia->is_video) {
if(in_array($singleMedia->video_url, $enclosures)) continue; // check if not added yet
$content .= '<video controls><source src="' . $singleMedia->video_url . '" type="video/mp4"></video><br>';
array_push($enclosures, $singleMedia->video_url);
} else {
if(in_array($singleMedia->display_url, $enclosures)) continue; // check if not added yet
$content .= '<a href="' . $singleMedia->display_url . '" target="_blank">';
$content .= '<img src="' . $singleMedia->display_url . '" alt="' . $postTitle . '" />';
$content .= '</a><br>';
array_push($enclosures, $singleMedia->display_url);
}
}
$content .= '<br>' . nl2br(htmlentities($textContent));
return [$content, $enclosures];
return array($content, $enclosures);
}
// returns Video post's contents and enclosures
protected function getInstagramVideoData($uri) {
$mediaInfo = $this->getSinglePostData($uri);
$textContent = $this->getTextContent($mediaInfo);
$content = '<video controls><source src="' . $mediaInfo->video_url . '" type="video/mp4"></video><br>';
$content .= '<br>' . nl2br(htmlentities($textContent));
return array($content, array($mediaInfo->video_url));
}
protected function getTextContent($media) {
$textContent = '(no text)';
//Process the first element, that isn't in the node graph
if (count($media->edge_media_to_caption->edges) > 0) {
$textContent = trim($media->edge_media_to_caption->edges[0]->node->text);
}
return $textContent;
}
protected function getSinglePostData($uri) {
$shortcode = explode('/', $uri)[4];
$data = getContents(self::URI .
'graphql/query/?query_hash=' .
self::SHORTCODE_QUERY_HASH .
'&variables={"shortcode"%3A"' .
$shortcode .
'"}');
return json_decode($data)->data->shortcode_media;
}
protected function getInstagramJSON($uri) {
$html = getContents($uri)
or returnServerError('Could not request Instagram.');
$scriptRegex = '/window\._sharedData = (.*);<\/script>/';
if(!is_null($this->getInput('u'))) {
preg_match($scriptRegex, $html, $matches, PREG_OFFSET_CAPTURE, 0);
$userId = $this->getInstagramUserId($this->getInput('u'));
return json_decode($matches[1][0]);
$data = getContents(self::URI .
'graphql/query/?query_hash=' .
self::USER_QUERY_HASH .
'&variables={"id"%3A"' .
$userId .
'"%2C"first"%3A10}');
return json_decode($data);
} elseif(!is_null($this->getInput('h'))) {
$data = getContents(self::URI .
'graphql/query/?query_hash=' .
self::TAG_QUERY_HASH .
'&variables={"tag_name"%3A"' .
$this->getInput('h') .
'"%2C"first"%3A10}');
return json_decode($data);
} else {
$html = getContents($uri)
or returnServerError('Could not request Instagram.');
$scriptRegex = '/window\._sharedData = (.*);<\/script>/';
preg_match($scriptRegex, $html, $matches, PREG_OFFSET_CAPTURE, 0);
return json_decode($matches[1][0]);
}
}

View File

@ -1,8 +1,7 @@
<?php
/**
* This class implements a bridge for http://www.instructables.com, supporting
* general feeds and feeds by category. Instructables doesn't support HTTPS as
* of now (23.06.2018), so all connections are insecure!
* general feeds and feeds by category.
*
* Remarks:
* - For some reason it is very important to have the category URI end with a
@ -13,7 +12,7 @@
*/
class InstructablesBridge extends BridgeAbstract {
const NAME = 'Instructables Bridge';
const URI = 'http://www.instructables.com';
const URI = 'https://www.instructables.com';
const DESCRIPTION = 'Returns general feeds and feeds by category';
const MAINTAINER = 'logmanoriginal';
const PARAMETERS = array(
@ -22,219 +21,201 @@ class InstructablesBridge extends BridgeAbstract {
'name' => 'Category',
'type' => 'list',
'values' => array(
'Play' => array(
'All' => '/play/',
'KNEX' => '/play/knex/',
'Offbeat' => '/play/offbeat/',
'Lego' => '/play/lego/',
'Airsoft' => '/play/airsoft/',
'Card Games' => '/play/card-games/',
'Guitars' => '/play/guitars/',
'Instruments' => '/play/instruments/',
'Magic Tricks' => '/play/magic-tricks/',
'Minecraft' => '/play/minecraft/',
'Music' => '/play/music/',
'Nerf' => '/play/nerf/',
'Nintendo' => '/play/nintendo/',
'Office Supplies' => '/play/office-supplies/',
'Paintball' => '/play/paintball/',
'Paper Airplanes' => '/play/paper-airplanes/',
'Party Tricks' => '/play/party-tricks/',
'PlayStation' => '/play/playstation/',
'Pranks and Humor' => '/play/pranks-and-humor/',
'Puzzles' => '/play/puzzles/',
'Siege Engines' => '/play/siege-engines/',
'Sports' => '/play/sports/',
'Table Top' => '/play/table-top/',
'Toys' => '/play/toys/',
'Video Games' => '/play/video-games/',
'Wii' => '/play/wii/',
'Xbox' => '/play/xbox/',
'Yo-Yo' => '/play/yo-yo/',
),
'Craft' => array(
'All' => '/craft/',
'Art' => '/craft/art/',
'Sewing' => '/craft/sewing/',
'Paper' => '/craft/paper/',
'Jewelry' => '/craft/jewelry/',
'Fashion' => '/craft/fashion/',
'Books & Journals' => '/craft/books-and-journals/',
'Cards' => '/craft/cards/',
'Clay' => '/craft/clay/',
'Duct Tape' => '/craft/duct-tape/',
'Embroidery' => '/craft/embroidery/',
'Felt' => '/craft/felt/',
'Fiber Arts' => '/craft/fiber-arts/',
'Gifts & Wrapping' => '/craft/gifts-and-wrapping/',
'Knitting & Crocheting' => '/craft/knitting-and-crocheting/',
'Leather' => '/craft/leather/',
'Mason Jars' => '/craft/mason-jars/',
'No-Sew' => '/craft/no-sew/',
'Parties & Weddings' => '/craft/parties-and-weddings/',
'Print Making' => '/craft/print-making/',
'Soap' => '/craft/soap/',
'Wallets' => '/craft/wallets/',
),
'Technology' => array(
'All' => '/technology/',
'Electronics' => '/technology/electronics/',
'Arduino' => '/technology/arduino/',
'Photography' => '/technology/photography/',
'Leds' => '/technology/leds/',
'Science' => '/technology/science/',
'Reuse' => '/technology/reuse/',
'Apple' => '/technology/apple/',
'Computers' => '/technology/computers/',
'3D Printing' => '/technology/3D-Printing/',
'Robots' => '/technology/robots/',
'Art' => '/technology/art/',
'Assistive Tech' => '/technology/assistive-technology/',
'Audio' => '/technology/audio/',
'Clocks' => '/technology/clocks/',
'CNC' => '/technology/cnc/',
'Digital Graphics' => '/technology/digital-graphics/',
'Gadgets' => '/technology/gadgets/',
'Kits' => '/technology/kits/',
'Laptops' => '/technology/laptops/',
'Lasers' => '/technology/lasers/',
'Linux' => '/technology/linux/',
'Microcontrollers' => '/technology/microcontrollers/',
'Microsoft' => '/technology/microsoft/',
'Mobile' => '/technology/mobile/',
'Raspberry Pi' => '/technology/raspberry-pi/',
'Remote Control' => '/technology/remote-control/',
'Sensors' => '/technology/sensors/',
'Software' => '/technology/software/',
'Soldering' => '/technology/soldering/',
'Speakers' => '/technology/speakers/',
'Steampunk' => '/technology/steampunk/',
'Tools' => '/technology/tools/',
'USB' => '/technology/usb/',
'Wearables' => '/technology/wearables/',
'Websites' => '/technology/websites/',
'Wireless' => '/technology/wireless/',
'Circuits' => array(
'All' => '/circuits/',
'Apple' => '/circuits/apple/projects/',
'Arduino' => '/circuits/arduino/projects/',
'Art' => '/circuits/art/projects/',
'Assistive Tech' => '/circuits/assistive-tech/projects/',
'Audio' => '/circuits/audio/projects/',
'Cameras' => '/circuits/cameras/projects/',
'Clocks' => '/circuits/clocks/projects/',
'Computers' => '/circuits/computers/projects/',
'Electronics' => '/circuits/electronics/projects/',
'Gadgets' => '/circuits/gadgets/projects/',
'Lasers' => '/circuits/lasers/projects/',
'LEDs' => '/circuits/leds/projects/',
'Linux' => '/circuits/linux/projects/',
'Microcontrollers' => '/circuits/microcontrollers/projects/',
'Microsoft' => '/circuits/microsoft/projects/',
'Mobile' => '/circuits/mobile/projects/',
'Raspberry Pi' => '/circuits/raspberry-pi/projects/',
'Remote Control' => '/circuits/remote-control/projects/',
'Reuse' => '/circuits/reuse/projects/',
'Robots' => '/circuits/robots/projects/',
'Sensors' => '/circuits/sensors/projects/',
'Software' => '/circuits/software/projects/',
'Soldering' => '/circuits/soldering/projects/',
'Speakers' => '/circuits/speakers/projects/',
'Tools' => '/circuits/tools/projects/',
'USB' => '/circuits/usb/projects/',
'Wearables' => '/circuits/wearables/projects/',
'Websites' => '/circuits/websites/projects/',
'Wireless' => '/circuits/wireless/projects/',
),
'Workshop' => array(
'All' => '/workshop/',
'Woodworking' => '/workshop/woodworking/',
'Tools' => '/workshop/tools/',
'Gardening' => '/workshop/gardening/',
'Cars' => '/workshop/cars/',
'Metalworking' => '/workshop/metalworking/',
'Cardboard' => '/workshop/cardboard/',
'Electric Vehicles' => '/workshop/electric-vehicles/',
'Energy' => '/workshop/energy/',
'Furniture' => '/workshop/furniture/',
'Home Improvement' => '/workshop/home-improvement/',
'Home Theater' => '/workshop/home-theater/',
'Hydroponics' => '/workshop/hydroponics/',
'Laser Cutting' => '/workshop/laser-cutting/',
'Lighting' => '/workshop/lighting/',
'Molds & Casting' => '/workshop/molds-and-casting/',
'Motorcycles' => '/workshop/motorcycles/',
'Organizing' => '/workshop/organizing/',
'Pallets' => '/workshop/pallets/',
'Repair' => '/workshop/repair/',
'Shelves' => '/workshop/shelves/',
'Solar' => '/workshop/solar/',
'Workbenches' => '/workshop/workbenches/',
'3D Printing' => '/workshop/3d-printing/projects/',
'Cars' => '/workshop/cars/projects/',
'CNC' => '/workshop/cnc/projects/',
'Electric Vehicles' => '/workshop/electric-vehicles/projects/',
'Energy' => '/workshop/energy/projects/',
'Furniture' => '/workshop/furniture/projects/',
'Home Improvement' => '/workshop/home-improvement/projects/',
'Home Theater' => '/workshop/home-theater/projects/',
'Hydroponics' => '/workshop/hydroponics/projects/',
'Knives' => '/workshop/knives/projects/',
'Laser Cutting' => '/workshop/laser-cutting/projects/',
'Lighting' => '/workshop/lighting/projects/',
'Metalworking' => '/workshop/metalworking/projects/',
'Molds & Casting' => '/workshop/molds-and-casting/projects/',
'Motorcycles' => '/workshop/motorcycles/projects/',
'Organizing' => '/workshop/organizing/projects/',
'Pallets' => '/workshop/pallets/projects/',
'Repair' => '/workshop/repair/projects/',
'Science' => '/workshop/science/projects/',
'Shelves' => '/workshop/shelves/projects/',
'Solar' => '/workshop/solar/projects/',
'Tools' => '/workshop/tools/projects/',
'Woodworking' => '/workshop/woodworking/projects/',
'Workbenches' => '/workshop/workbenches/projects/',
),
'Home' => array(
'All' => '/home/',
'Halloween' => '/home/halloween/',
'Decorating' => '/home/decorating/',
'Organizing' => '/home/organizing/',
'Pets' => '/home/pets/',
'Life Hacks' => '/home/life-hacks/',
'Beauty' => '/home/beauty/',
'Christmas' => '/home/christmas/',
'Cleaning' => '/home/cleaning/',
'Education' => '/home/education/',
'Finances' => '/home/finances/',
'Gardening' => '/home/gardening/',
'Green' => '/home/green/',
'Health' => '/home/health/',
'Hiding Places' => '/home/hiding-places/',
'Holidays' => '/home/holidays/',
'Homesteading' => '/home/homesteading/',
'Kids' => '/home/kids/',
'Kitchen' => '/home/kitchen/',
'Life Skills' => '/home/life-skills/',
'Parenting' => '/home/parenting/',
'Pest Control' => '/home/pest-control/',
'Relationships' => '/home/relationships/',
'Reuse' => '/home/reuse/',
'Travel' => '/home/travel/',
'Craft' => array(
'All' => '/craft/',
'Art' => '/craft/art/projects/',
'Books & Journals' => '/craft/books-and-journals/projects/',
'Cardboard' => '/craft/cardboard/projects/',
'Cards' => '/craft/cards/projects/',
'Clay' => '/craft/clay/projects/',
'Costumes & Cosplay' => '/craft/costumes-and-cosplay/projects/',
'Digital Graphics' => '/craft/digital-graphics/projects/',
'Duct Tape' => '/craft/duct-tape/projects/',
'Embroidery' => '/craft/embroidery/projects/',
'Fashion' => '/craft/fashion/projects/',
'Felt' => '/craft/felt/projects/',
'Fiber Arts' => '/craft/fiber-arts/projects/',
'Gift Wrapping' => '/craft/gift-wrapping/projects/',
'Jewelry' => '/craft/jewelry/projects/',
'Knitting & Crochet' => '/craft/knitting-and-crochet/projects/',
'Leather' => '/craft/leather/projects/',
'Mason Jars' => '/craft/mason-jars/projects/',
'No-Sew' => '/craft/no-sew/projects/',
'Paper' => '/craft/paper/projects/',
'Parties & Weddings' => '/craft/parties-and-weddings/projects/',
'Photography' => '/craft/photography/projects/',
'Printmaking' => '/craft/printmaking/projects/',
'Reuse' => '/craft/reuse/projects/',
'Sewing' => '/craft/sewing/projects/',
'Soapmaking' => '/craft/soapmaking/projects/',
'Wallets' => '/craft/wallets/projects/',
),
'Cooking' => array(
'All' => '/cooking/',
'Bacon' => '/cooking/bacon/projects/',
'BBQ & Grilling' => '/cooking/bbq-and-grilling/projects/',
'Beverages' => '/cooking/beverages/projects/',
'Bread' => '/cooking/bread/projects/',
'Breakfast' => '/cooking/breakfast/projects/',
'Cake' => '/cooking/cake/projects/',
'Candy' => '/cooking/candy/projects/',
'Canning & Preserving' => '/cooking/canning-and-preserving/projects/',
'Cocktails & Mocktails' => '/cooking/cocktails-and-mocktails/projects/',
'Coffee' => '/cooking/coffee/projects/',
'Cookies' => '/cooking/cookies/projects/',
'Cupcakes' => '/cooking/cupcakes/projects/',
'Dessert' => '/cooking/dessert/projects/',
'Homebrew' => '/cooking/homebrew/projects/',
'Main Course' => '/cooking/main-course/projects/',
'Pasta' => '/cooking/pasta/projects/',
'Pie' => '/cooking/pie/projects/',
'Pizza' => '/cooking/pizza/projects/',
'Salad' => '/cooking/salad/projects/',
'Sandwiches' => '/cooking/sandwiches/projects/',
'Snacks & Appetizers' => '/cooking/snacks-and-appetizers/projects/',
'Soups & Stews' => '/cooking/soups-and-stews/projects/',
'Vegetarian & Vegan' => '/cooking/vegetarian-and-vegan/projects/',
),
'Living' => array(
'All' => '/living/',
'Beauty' => '/living/beauty/projects/',
'Christmas' => '/living/christmas/projects/',
'Cleaning' => '/living/cleaning/projects/',
'Decorating' => '/living/decorating/projects/',
'Education' => '/living/education/projects/',
'Gardening' => '/living/gardening/projects/',
'Halloween' => '/living/halloween/projects/',
'Health' => '/living/health/projects/',
'Hiding Places' => '/living/hiding-places/projects/',
'Holidays' => '/living/holidays/projects/',
'Homesteading' => '/living/homesteading/projects/',
'Kids' => '/living/kids/projects/',
'Kitchen' => '/living/kitchen/projects/',
'LEGO & KNEX' => '/living/lego-and-knex/projects/',
'Life Hacks' => '/living/life-hacks/projects/',
'Music' => '/living/music/projects/',
'Office Supply Hacks' => '/living/office-supply-hacks/projects/',
'Organizing' => '/living/organizing/projects/',
'Pest Control' => '/living/pest-control/projects/',
'Pets' => '/living/pets/projects/',
'Pranks, Tricks, & Humor' => '/living/pranks-tricks-and-humor/projects/',
'Relationships' => '/living/relationships/projects/',
'Toys & Games' => '/living/toys-and-games/projects/',
'Travel' => '/living/travel/projects/',
'Video Games' => '/living/video-games/projects/',
),
'Outside' => array(
'All' => '/outside/',
'Bikes' => '/outside/bikes/',
'Survival' => '/outside/survival/',
'Backyard' => '/outside/backyard/',
'Beach' => '/outside/beach/',
'Birding' => '/outside/birding/',
'Boats' => '/outside/boats/',
'Camping' => '/outside/camping/',
'Climbing' => '/outside/climbing/',
'Fire' => '/outside/fire/',
'Fishing' => '/outside/fishing/',
'Hunting' => '/outside/hunting/',
'Kites' => '/outside/kites/',
'Knives' => '/outside/knives/',
'Knots' => '/outside/knots/',
'Paracord' => '/outside/paracord/',
'Rockets' => '/outside/rockets/',
'Skateboarding' => '/outside/skateboarding/',
'Snow' => '/outside/snow/',
'Water' => '/outside/water/',
'Backyard' => '/outside/backyard/projects/',
'Beach' => '/outside/beach/projects/',
'Bikes' => '/outside/bikes/projects/',
'Birding' => '/outside/birding/projects/',
'Boats' => '/outside/boats/projects/',
'Camping' => '/outside/camping/projects/',
'Climbing' => '/outside/climbing/projects/',
'Fire' => '/outside/fire/projects/',
'Fishing' => '/outside/fishing/projects/',
'Hunting' => '/outside/hunting/projects/',
'Kites' => '/outside/kites/projects/',
'Knots' => '/outside/knots/projects/',
'Launchers' => '/outside/launchers/projects/',
'Paracord' => '/outside/paracord/projects/',
'Rockets' => '/outside/rockets/projects/',
'Siege Engines' => '/outside/siege-engines/projects/',
'Skateboarding' => '/outside/skateboarding/projects/',
'Snow' => '/outside/snow/projects/',
'Sports' => '/outside/sports/projects/',
'Survival' => '/outside/survival/projects/',
'Water' => '/outside/water/projects/',
),
'Food' => array(
'All' => '/food/',
'Dessert' => '/food/dessert/',
'Snacks & Appetizers' => '/food/snacks-and-appetizers/',
'Bacon' => '/food/bacon/',
'BBQ & Grilling' => '/food/bbq-and-grilling/',
'Beverages' => '/food/beverages/',
'Bread' => '/food/bread/',
'Breakfast' => '/food/breakfast/',
'Cake' => '/food/cake/',
'Candy' => '/food/candy/',
'Canning & Preserves' => '/food/canning-and-preserves/',
'Cocktails & Mocktails' => '/food/cocktails-and-mocktails/',
'Coffee' => '/food/coffee/',
'Cookies' => '/food/cookies/',
'Cupcakes' => '/food/cupcakes/',
'Homebrew' => '/food/homebrew/',
'Main Course' => '/food/main-course/',
'Pasta' => '/food/pasta/',
'Pie' => '/food/pie/',
'Pizza' => '/food/pizza/',
'Salad' => '/food/salad/',
'Sandwiches' => '/food/sandwiches/',
'Soups & Stews' => '/food/soups-and-stews/',
'Vegetarian & Vegan' => '/food/vegetarian-and-vegan/',
'Makeymakey' => array(
'All' => '/makeymakey/',
'Makey Makey on Instructables' => '/makeymakey/',
),
'Teachers' => array(
'All' => '/teachers/',
'ELA' => '/teachers/ela/projects/',
'Math' => '/teachers/math/projects/',
'Science' => '/teachers/science/projects/',
'Social Studies' => '/teachers/social-studies/projects/',
'Engineering' => '/teachers/engineering/projects/',
'Coding' => '/teachers/coding/projects/',
'Electronics' => '/teachers/electronics/projects/',
'Robotics' => '/teachers/robotics/projects/',
'Arduino' => '/teachers/arduino/projects/',
'CNC' => '/teachers/cnc/projects/',
'Laser Cutting' => '/teachers/laser-cutting/projects/',
'3D Printing' => '/teachers/3d-printing/projects/',
'3D Design' => '/teachers/3d-design/projects/',
'Art' => '/teachers/art/projects/',
'Music' => '/teachers/music/projects/',
'Theatre' => '/teachers/theatre/projects/',
'Wood Shop' => '/teachers/wood-shop/projects/',
'Metal Shop' => '/teachers/metal-shop/projects/',
'Resources' => '/teachers/resources/projects/',
),
'Costumes' => array(
'All' => '/costumes/',
'Props' => '/costumes/props-and-accessories/',
'Animals' => '/costumes/animals/',
'Comics' => '/costumes/comics/',
'Fantasy' => '/costumes/fantasy/',
'For Kids' => '/costumes/for-kids/',
'For Pets' => '/costumes/for-pets/',
'Funny' => '/costumes/funny/',
'Games' => '/costumes/games/',
'Historic & Futuristic' => '/costumes/historic-and-futuristic/',
'Makeup' => '/costumes/makeup/',
'Masks' => '/costumes/masks/',
'Scary' => '/costumes/scary/',
'TV & Movies' => '/costumes/tv-and-movies/',
'Weapons & Armor' => '/costumes/weapons-and-armor/',
)
),
'title' => 'Select your category (required)',
'defaultValue' => 'Technology'
'defaultValue' => 'Circuits'
),
'filter' => array(
'name' => 'Filter',
@ -252,65 +233,70 @@ class InstructablesBridge extends BridgeAbstract {
)
);
private $uri;
public function collectData() {
// Enable the following line to get the category list (dev mode)
// $this->listCategories();
$this->uri = static::URI;
$html = getSimpleHTMLDOM($this->getURI())
or returnServerError('Error loading category ' . $this->getURI());
$html = defaultLinkTo($html, $this->getURI());
switch($this->queriedContext) {
case 'Category': $this->uri .= $this->getInput('category') . $this->getInput('filter');
}
$covers = $html->find('
.category-projects-list > div,
.category-landing-projects-list > div,
');
$html = getSimpleHTMLDOM($this->uri)
or returnServerError('Error loading category ' . $this->uri);
foreach($html->find('ul.explore-covers-list li') as $cover) {
foreach($covers as $cover) {
$item = array();
$item['uri'] = static::URI . $cover->find('a.cover-image', 0)->href;
$item['title'] = $cover->find('.title', 0)->innertext;
$item['uri'] = $cover->find('a.ible-title', 0)->href;
$item['title'] = $cover->find('a.ible-title', 0)->innertext;
$item['author'] = $this->getCategoryAuthor($cover);
$item['content'] = '<a href='
. $item['uri']
. '><img src='
. $cover->find('a.cover-image img', 0)->src
. $cover->find('img', 0)->getAttribute('data-src')
. '></a>';
$image = str_replace('.RECTANGLE1', '.LARGE', $cover->find('a.cover-image img', 0)->src);
$item['enclosures'] = [$image];
$item['enclosures'][] = str_replace(
'.RECTANGLE1',
'.LARGE',
$cover->find('img', 0)->getAttribute('data-src')
);
$this->items[] = $item;
}
}
public function getName() {
if(!is_null($this->getInput('category'))
&& !is_null($this->getInput('filter'))) {
foreach(self::PARAMETERS[$this->queriedContext]['category']['values'] as $key => $value) {
$subcategory = array_search($this->getInput('category'), $value);
switch($this->queriedContext) {
case 'Category': {
foreach(self::PARAMETERS[$this->queriedContext]['category']['values'] as $key => $value) {
$subcategory = array_search($this->getInput('category'), $value);
if($subcategory !== false)
break;
}
if($subcategory !== false)
break;
}
$filter = array_search(
$this->getInput('filter'),
self::PARAMETERS[$this->queriedContext]['filter']['values']
);
$filter = array_search(
$this->getInput('filter'),
self::PARAMETERS[$this->queriedContext]['filter']['values']
);
return $subcategory . ' (' . $filter . ') - ' . static::NAME;
return $subcategory . ' (' . $filter . ') - ' . static::NAME;
} break;
}
return parent::getName();
}
public function getURI() {
if(!is_null($this->getInput('category'))
&& !is_null($this->getInput('filter'))) {
return $this->uri;
switch($this->queriedContext) {
case 'Category': {
return self::URI
. $this->getInput('category')
. $this->getInput('filter');
} break;
}
return parent::getURI();
@ -321,24 +307,32 @@ class InstructablesBridge extends BridgeAbstract {
* parameters list)
*/
private function listCategories(){
// Use arbitrary category to receive full list
$html = getSimpleHTMLDOM(self::URI . '/technology/');
foreach($html->find('.channel a') as $channel) {
$name = html_entity_decode(trim($channel->innertext));
// Use home page to acquire main categories
$html = getSimpleHTMLDOM(self::URI);
$html = defaultLinkTo($html, self::URI);
// Remove unwanted entities
$name = str_replace("'", '', $name);
$name = str_replace('&#39;', '', $name);
foreach($html->find('.home-content-explore-link') as $category) {
$uri = $channel->href;
// Use arbitrary category to receive full list
$html = getSimpleHTMLDOM($category->href);
$category = explode('/', $uri)[1];
foreach($html->find('.channel-thumbnail a') as $channel) {
$name = html_entity_decode(trim($channel->title));
if(!isset($categories)
|| !array_key_exists($category, $categories)
|| !in_array($uri, $categories[$category]))
$categories[$category][$name] = $uri;
// Remove unwanted entities
$name = str_replace("'", '', $name);
$name = str_replace('&#39;', '', $name);
$uri = $channel->href;
$category_name = explode('/', $uri)[1];
if(!isset($categories)
|| !array_key_exists($category_name, $categories)
|| !in_array($uri, $categories[$category_name]))
$categories[$category_name][$name] = $uri;
}
}
// Build PHP array manually
@ -360,9 +354,9 @@ class InstructablesBridge extends BridgeAbstract {
*/
private function getCategoryAuthor($cover) {
return '<a href='
. static::URI . $cover->find('span.author a', 0)->href
. $cover->find('.ible-author a', 0)->href
. '>'
. $cover->find('span.author a', 0)->innertext
. $cover->find('.ible-author a', 0)->innertext
. '</a>';
}
}

View File

@ -0,0 +1,293 @@
<?php
class InternetArchiveBridge extends BridgeAbstract {
const NAME = 'Internet Archive Bridge';
const URI = 'https://archive.org';
const DESCRIPTION = 'Returns newest uploads, posts and more from an account';
const MAINTAINER = 'VerifiedJoseph';
const PARAMETERS = array(
'Account' => array(
'username' => array(
'name' => 'Username',
'type' => 'text',
'required' => true,
'exampleValue' => '@verifiedjoseph',
),
'content' => array(
'name' => 'Content',
'type' => 'list',
'values' => array(
'Uploads' => 'uploads',
'Posts' => 'posts',
'Reviews' => 'reviews',
'Collections' => 'collections',
'Web Archives' => 'web-archive',
),
'defaultValue' => 'uploads',
)
)
);
const CACHE_TIMEOUT = 900; // 15 mins
private $skipClasses = array(
'item-ia mobile-header hidden-tiles',
'item-ia account-ia'
);
public function collectData() {
$html = getSimpleHTMLDOM($this->getURI())
or returnServerError('Could not request: ' . $this->getURI());
$html = defaultLinkTo($html, $this->getURI());
if ($this->getInput('content') !== 'posts') {
$detailsDivNumber = 0;
foreach ($html->find('div.results > div[data-id]') as $index => $result) {
$item = array();
if (in_array($result->class, $this->skipClasses)) {
continue;
}
switch($result->class) {
case 'item-ia':
switch($this->getInput('content')) {
case 'reviews':
$item = $this->processReview($result);
break;
case 'uploads':
$item = $this->processUpload($result);
break;
}
break;
case 'item-ia url-item':
$item = $this->processWebArchives($result);
break;
case 'item-ia collection-ia':
$item = $this->processCollection($result);
break;
}
if ($this->getInput('content') !== 'reviews') {
$hiddenDetails = $this->processHiddenDetails($html, $detailsDivNumber, $item);
$this->items[] = array_merge($item, $hiddenDetails);
} else {
$this->items[] = $item;
}
$detailsDivNumber++;
}
}
if ($this->getInput('content') === 'posts') {
$this->items = $this->processPosts($html);
}
}
public function getURI() {
if (!is_null($this->getInput('username')) && !is_null($this->getInput('content'))) {
return self::URI . '/details/' . $this->processUsername() . '&tab=' . $this->getInput('content');
}
return parent::getURI();
}
public function getName() {
if (!is_null($this->getInput('username')) && !is_null($this->getInput('content'))) {
$contentValues = array_flip(self::PARAMETERS['Account']['content']['values']);
return $contentValues[$this->getInput('content')] . ' - '
. $this->processUsername() . ' - Internet Archive';
}
return parent::getName();
}
private function processUsername() {
if (substr($this->getInput('username'), 0, 1) !== '@') {
return '@' . $this->getInput('username');
}
return $this->getInput('username');
}
private function processUpload($result) {
$item = array();
$collection = $result->find('a.stealth', 0);
$collectionLink = self::URI . $collection->href;
$collectionTitle = $collection->find('div.item-parent-ttl', 0)->plaintext;
$item['title'] = trim($result->find('div.ttl', 0)->innertext);
$item['timestamp'] = strtotime($result->find('div.hidden-tiles.pubdate.C.C3', 0)->children(0)->plaintext);
$item['uri'] = $result->find('div.item-ttl.C.C2 > a', 0)->href;
if ($result->find('div.by.C.C4', 0)->children(2)) {
$item['author'] = $result->find('div.by.C.C4', 0)->children(2)->plaintext;
}
$item['content'] = <<<EOD
<p>Media Type: {$result->attr['data-mediatype']}<br>
Collection: <a href="{$collectionLink}">{$collectionTitle}</a></p>
EOD;
$item['enclosures'][] = self::URI . $result->find('img.item-img', 0)->source;
return $item;
}
private function processReview($result) {
$item = array();
$item['title'] = trim($result->find('div.ttl', 0)->innertext);
$item['timestamp'] = strtotime($result->find('div.hidden-tiles.pubdate.C.C3', 0)->children(0)->plaintext);
$item['uri'] = $result->find('div.review-title', 0)->children(0)->href;
if ($result->find('div.by.C.C4', 0)->children(2)) {
$item['author'] = $result->find('div.by.C.C4', 0)->children(2)->plaintext;
}
$item['content'] = <<<EOD
<p><strong>Subject: {$result->find('div.review-title', 0)->plaintext}</strong></p>
<p>{$result->find('div.hidden-lists.review' , 0)->children(1)->plaintext}</p>
EOD;
$item['enclosures'][] = self::URI . $result->find('img.item-img', 0)->source;
return $item;
}
private function processWebArchives($result) {
$item = array();
$item['title'] = trim($result->find('div.ttl', 0)->plaintext);
$item['timestamp'] = strtotime($result->find('div.hidden-lists', 0)->children(0)->plaintext);
$item['uri'] = $result->find('div.item-ttl.C.C2 > a', 0)->href;
$item['content'] = <<<EOD
{$this->processUsername()} archived <a href="{$item['uri']}">{$result->find('div.ttl', 0)->plaintext}</a>
EOD;
$item['enclosures'][] = $result->find('img.item-img', 0)->source;
return $item;
}
private function processCollection($result) {
$item = array();
$title = trim($result->find('div.collection-title.C.C2', 0)->children(0)->plaintext);
$itemCount = strtolower(trim($result->find('div.num-items.topinblock', 0)->plaintext));
$item['title'] = $title . ' (' . $itemCount . ')';
$item['timestamp'] = strtotime($result->find('div.hidden-tiles.pubdate.C.C3', 0)->children(0)->plaintext);
$item['uri'] = $result->find('div.collection-title.C.C2 > a', 0)->href;
$item['content'] = '';
if ($result->find('img.item-img', 0)) {
$item['enclosures'][] = self::URI . $result->find('img.item-img', 0)->source;
}
return $item;
}
private function processHiddenDetails($html, $detailsDivNumber, $item) {
$description = '';
if ($html->find('div.details-ia.hidden-tiles', $detailsDivNumber)) {
$detailsDiv = $html->find('div.details-ia.hidden-tiles', $detailsDivNumber);
if ($detailsDiv->find('div.C234', 0)->children(0)) {
$description = $detailsDiv->find('div.C234', 0)->children(0)->plaintext;
$detailsDiv->find('div.C234', 0)->children(0)->innertext = '';
}
$topics = trim($detailsDiv->find('div.C234', 0)->plaintext);
if (!empty($topics)) {
$topics = trim($detailsDiv->find('div.C234', 0)->plaintext);
$topics = trim(substr($topics, 7));
$item['categories'] = explode(',', $topics);
}
$item['content'] = '<p>' . $description . '</p>' . $item['content'];
}
return $item;
}
private function processPosts($html) {
$items = array();
foreach ($html->find('table.forumTable > tr') as $index => $tr) {
$item = array();
if ($index === 0) {
continue;
}
$item['title'] = $tr->find('td', 0)->plaintext;
$item['timestamp'] = strtotime($tr->find('td', 4)->children(0)->plaintext);
$item['uri'] = $tr->find('td', 0)->children(0)->href;
$formLink = <<<EOD
<a href="{$tr->find('td', 2)->children(0)->href}">{$tr->find('td', 2)->children(0)->plaintext}</a>
EOD;
$postDate = $tr->find('td', 4)->children(0)->plaintext;
$postPageHtml = getSimpleHTMLDOMCached($item['uri'], 3600)
or returnServerError('Could not request: ' . $item['uri']);
$postPageHtml = defaultLinkTo($postPageHtml, $this->getURI());
$post = $postPageHtml->find('div.box.well.well-sm', 0);
$parentLink = '';
$replyLink = <<<EOD
<a href="{$post->find('a', 0)->href}">Reply</a>
EOD;
if ($post->find('a', 1)->innertext = 'See parent post') {
$parentLink = <<<EOD
<a href="{$post->find('a', 1)->href}">View parent post</a>
EOD;
}
$post->find('h1', 0)->outertext = '';
$post->find('h2', 0)->outertext = '';
$item['content'] = <<<EOD
<p>{$post->innertext}</p>{$replyLink} - {$parentLink} - Posted in {$formLink} on {$postDate}
EOD;
$items[] = $item;
if (count($items) >= 10) {
break;
}
}
return $items;
}
}

View File

@ -19,28 +19,6 @@ class JapanExpoBridge extends BridgeAbstract {
public function collectData(){
function frenchPubDateToTimestamp($date_to_parse) {
return strtotime(
strtr(
strtolower(str_replace('Publié le ', '', $date_to_parse)),
array(
'janvier' => 'jan',
'février' => 'feb',
'mars' => 'march',
'avril' => 'apr',
'mai' => 'may',
'juin' => 'jun',
'juillet' => 'jul',
'août' => 'aug',
'septembre' => 'sep',
'octobre' => 'oct',
'novembre' => 'nov',
'décembre' => 'dec'
)
)
);
}
$convert_article_images = function($matches){
if(is_array($matches) && count($matches) > 1) {
return '<img src="' . $matches[1] . '" />';
@ -82,7 +60,7 @@ class JapanExpoBridge extends BridgeAbstract {
$content = $headings . $article;
} else {
$date_text = $element->find('span.date', 0)->plaintext;
$timestamp = frenchPubDateToTimestamp($date_text);
$timestamp = $this->frenchPubDateToTimestamp($date_text);
$title = trim($element->find('span._title', 0)->plaintext);
$content = '<img src="'
. $thumbnail
@ -103,4 +81,26 @@ class JapanExpoBridge extends BridgeAbstract {
$count++;
}
}
private function frenchPubDateToTimestamp($date_to_parse) {
return strtotime(
strtr(
strtolower(str_replace('Publié le ', '', $date_to_parse)),
array(
'janvier' => 'jan',
'février' => 'feb',
'mars' => 'march',
'avril' => 'apr',
'mai' => 'may',
'juin' => 'jun',
'juillet' => 'jul',
'août' => 'aug',
'septembre' => 'sep',
'octobre' => 'oct',
'novembre' => 'nov',
'décembre' => 'dec'
)
)
);
}
}

View File

@ -5,7 +5,7 @@ class KonachanBridge extends MoebooruBridge {
const MAINTAINER = 'mitsukarenai';
const NAME = 'Konachan';
const URI = 'http://konachan.com/';
const URI = 'https://konachan.com/';
const DESCRIPTION = 'Returns images from given page';
}

View File

@ -24,6 +24,22 @@ class KununuBridge extends BridgeAbstract {
'type' => 'checkbox',
'exampleValue' => 'checked',
'title' => 'Activate to load full article'
),
'include_ratings' => array(
'name' => 'Include ratings',
'type' => 'checkbox',
'title' => 'Activate to include ratings in the feed'
),
'include_benefits' => array(
'name' => 'Include benefits',
'type' => 'checkbox',
'title' => 'Activate to include benefits in the feed'
),
'limit' => array(
'name' => 'Limit',
'type' => 'number',
'defaultValue' => 3,
'title' => "Maximum number of items to return in the feed.\n0 = unlimited"
)
),
array(
@ -98,6 +114,8 @@ class KununuBridge extends BridgeAbstract {
$articles = $section->find('article')
or returnServerError('Unable to find articles!');
$limit = $this->getInput('limit') ?: 0;
// Go through all articles
foreach($articles as $article) {
@ -116,7 +134,7 @@ class KununuBridge extends BridgeAbstract {
$item = array();
$item['author'] = $this->extractArticleAuthorPosition($article);
$item['timestamp'] = strtotime($date);
$item['timestamp'] = strtotime($date->content);
$item['title'] = $rating->getAttribute('aria-label')
. ' : '
. strip_tags($summary->innertext);
@ -131,6 +149,8 @@ class KununuBridge extends BridgeAbstract {
$this->items[] = $item;
if ($limit > 0 && count($this->items) >= $limit) break;
}
}
@ -175,7 +195,32 @@ class KununuBridge extends BridgeAbstract {
$description = $article->find('[itemprop=reviewBody]', 0)
or returnServerError('Cannot find article description!');
return $description->innertext;
$retVal = $description->innertext;
if($this->getInput('include_ratings')
&& ($ratings = $article->find('.review-ratings .rating-group'))) {
$retVal .= (empty($retVal) ? '' : '<hr>') . '<table>';
foreach($ratings as $rating) {
$retVal .= <<<EOD
<tr>
<td>{$rating->find('.rating-title', 0)->plaintext}
<td>{$rating->find('.rating-badge', 0)->plaintext}
</tr>
EOD;
}
$retVal .= '</table>';
}
if($this->getInput('include_benefits')
&& ($benefits = $article->find('benefit'))) {
$retVal .= (empty($retVal) ? '' : '<hr>') . '<ul>';
foreach($benefits as $benefit) {
$retVal .= "<li>{$benefit->plaintext}</li>";
}
$retVal .= '</ul>';
}
return $retVal;
}
/**

View File

@ -0,0 +1,477 @@
<?php
class LaCentraleBridge extends BridgeAbstract {
const MAINTAINER = 'jacknumber';
const NAME = 'La Centrale';
const URI = 'https://www.lacentrale.fr/';
const DESCRIPTION = 'Returns most recent vehicules ads from LaCentrale';
const PARAMETERS = array( array(
'type' => array(
'name' => 'Type de véhicule',
'type' => 'list',
'values' => array(
'Voiture' => 'car',
'Camion/Pickup' => 'truck',
'Moto' => 'moto',
'Scooter' => 'scooter',
'Quad' => 'quad',
'Caravane/Camping-car' => 'mobileHome'
)
),
'brand' => array(
'name' => 'Marque',
'type' => 'list',
'values' => array(
'' => '',
'ABARTH' => 'ABARTH',
'AC' => 'AC',
'AIXAM' => 'AIXAM',
'ALFA ROMEO' => 'ALFA ROMEO',
'ALKE' => 'ALKE',
'ALPINA' => 'ALPINA',
'ALPINE' => 'ALPINE',
'AMC' => 'AMC',
'ANAIG' => 'ANAIG',
'APRILIA' => 'APRILIA',
'ARIEL' => 'ARIEL',
'ASTON MARTIN' => 'ASTON MARTIN',
'AUDI' => 'AUDI',
'AUSTIN HEALEY' => 'AUSTIN HEALEY',
'AUSTIN' => 'AUSTIN',
'AUTOBIANCHI' => 'AUTOBIANCHI',
'AVINTON' => 'AVINTON',
'BELLIER' => 'BELLIER',
'BENELLI' => 'BENELLI',
'BENTLEY' => 'BENTLEY',
'BETA' => 'BETA',
'BMW' => 'BMW',
'BOLLORE' => 'BOLLORE',
'BRIXTON' => 'BRIXTON',
'BUELL' => 'BUELL',
'BUGATTI' => 'BUGATTI',
'BUICK' => 'BUICK',
'BULLIT' => 'BULLIT',
'CADILLAC' => 'CADILLAC',
'CASALINI' => 'CASALINI',
'CATERHAM' => 'CATERHAM',
'CHATENET' => 'CHATENET',
'CHEVROLET' => 'CHEVROLET',
'CHRYSLER' => 'CHRYSLER',
'CHUNLAN' => 'CHUNLAN',
'CITROEN' => 'CITROEN',
'COURB' => 'COURB',
'CR&S' => 'CR&S',
'CUPRA' => 'CUPRA',
'CYCLONE' => 'CYCLONE',
'DACIA' => 'DACIA',
'DAELIM' => 'DAELIM',
'DAEWOO' => 'DAEWOO',
'DAF' => 'DAF',
'DAIHATSU' => 'DAIHATSU',
'DANGEL' => 'DANGEL',
'DATSUN' => 'DATSUN',
'DE SOTO' => 'DE SOTO',
'DE TOMASO' => 'DE TOMASO',
'DERBI' => 'DERBI',
'DEVINCI' => 'DEVINCI',
'DODGE' => 'DODGE',
'DONKERVOORT' => 'DONKERVOORT',
'DS' => 'DS',
'DUCATI' => 'DUCATI',
'DUCATY' => 'DUCATY',
'DUE' => 'DUE',
'ENFIELD' => 'ENFIELD',
'EXCALIBUR' => 'EXCALIBUR',
'FACEL VEGA' => 'FACEL VEGA',
'FANTIC MOTOR' => 'FANTIC MOTOR',
'FERRARI' => 'FERRARI',
'FIAT' => 'FIAT',
'FISKER' => 'FISKER',
'FORD' => 'FORD',
'FUSO' => 'FUSO',
'GAS GAS' => 'GAS GAS',
'GILERA' => 'GILERA',
'GMC' => 'GMC',
'GOWINN' => 'GOWINN',
'GRANDIN' => 'GRANDIN',
'HARLEY DAVIDSON' => 'HARLEY DAVIDSON',
'HOMMELL' => 'HOMMELL',
'HONDA' => 'HONDA',
'HUMMER' => 'HUMMER',
'HUSABERG' => 'HUSABERG',
'HUSQVARNA' => 'HUSQVARNA',
'HYOSUNG' => 'HYOSUNG',
'HYUNDAI' => 'HYUNDAI',
'INDIAN' => 'INDIAN',
'INFINITI' => 'INFINITI',
'INNOCENTI' => 'INNOCENTI',
'ISUZU' => 'ISUZU',
'IVECO' => 'IVECO',
'JAGUAR' => 'JAGUAR',
'JDM SIMPA' => 'JDM SIMPA',
'JEEP' => 'JEEP',
'JENSEN' => 'JENSEN',
'JIAYUAN' => 'JIAYUAN',
'KAWASAKI' => 'KAWASAKI',
'KEEWAY' => 'KEEWAY',
'KIA' => 'KIA',
'KSR' => 'KSR',
'KTM' => 'KTM',
'KYMCO' => 'KYMCO',
'LADA' => 'LADA',
'LAMBORGHINI' => 'LAMBORGHINI',
'LANCIA' => 'LANCIA',
'LAND ROVER' => 'LAND ROVER',
'LEXUS' => 'LEXUS',
'LIGIER' => 'LIGIER',
'LINCOLN' => 'LINCOLN',
'LONDON TAXI COMPANY' => 'LONDON TAXI COMPANY',
'LOTUS' => 'LOTUS',
'MAGPOWER' => 'MAGPOWER',
'MAN' => 'MAN',
'MASAI' => 'MASAI',
'MASERATI' => 'MASERATI',
'MASH' => 'MASH',
'MATRA' => 'MATRA',
'MAYBACH' => 'MAYBACH',
'MAZDA' => 'MAZDA',
'MCLAREN' => 'MCLAREN',
'MEGA' => 'MEGA',
'MERCEDES' => 'MERCEDES',
'MERCEDES-AMG' => 'MERCEDES-AMG',
'MERCURY' => 'MERCURY',
'MEYERS MANX' => 'MEYERS MANX',
'MG' => 'MG',
'MIA ELECTRIC' => 'MIA ELECTRIC',
'MICROCAR' => 'MICROCAR',
'MINAUTO' => 'MINAUTO',
'MINI' => 'MINI',
'MITSUBISHI' => 'MITSUBISHI',
'MORGAN' => 'MORGAN',
'MORRIS' => 'MORRIS',
'MOTO GUZZI' => 'MOTO GUZZI',
'MOTO MORINI' => 'MOTO MORINI',
'MOTOBECANE' => 'MOTOBECANE',
'MPM MOTORS' => 'MPM MOTORS',
'MV AGUSTA' => 'MV AGUSTA',
'NISSAN' => 'NISSAN',
'NORTON' => 'NORTON',
'NSU' => 'NSU',
'OLDSMOBILE' => 'OLDSMOBILE',
'OPEL' => 'OPEL',
'ORCAL' => 'ORCAL',
'OSSA' => 'OSSA',
'PACKARD' => 'PACKARD',
'PANTHER' => 'PANTHER',
'PEUGEOT' => 'PEUGEOT',
'PGO' => 'PGO',
'PIAGGIO' => 'PIAGGIO',
'PLYMOUTH' => 'PLYMOUTH',
'POLARIS' => 'POLARIS',
'PONTIAC' => 'PONTIAC',
'PORSCHE' => 'PORSCHE',
'REALM' => 'REALM',
'REGAL RAPTOR' => 'REGAL RAPTOR',
'RENAULT' => 'RENAULT',
'RIEJU' => 'RIEJU',
'ROLLS ROYCE' => 'ROLLS ROYCE',
'ROVER' => 'ROVER',
'ROYAL ENFIELD' => 'ROYAL ENFIELD',
'SAAB' => 'SAAB',
'SANTANA' => 'SANTANA',
'SCANIA' => 'SCANIA',
'SEAT' => 'SEAT',
'SECMA' => 'SECMA',
'SHELBY' => 'SHELBY',
'SHERCO' => 'SHERCO',
'SIMCA' => 'SIMCA',
'SKODA' => 'SKODA',
'SMART' => 'SMART',
'SPYKER' => 'SPYKER',
'SSANGYONG' => 'SSANGYONG',
'STUDEBAKER' => 'STUDEBAKER',
'SUBARU' => 'SUBARU',
'SUNBEAM' => 'SUNBEAM',
'SUZUKI' => 'SUZUKI',
'SWM' => 'SWM',
'SYM' => 'SYM',
'TALBOT SIMCA' => 'TALBOT SIMCA',
'TALBOT' => 'TALBOT',
'TEILHOL' => 'TEILHOL',
'TESLA' => 'TESLA',
'TM' => 'TM',
'TNT MOTOR' => 'TNT MOTOR',
'TOYOTA' => 'TOYOTA',
'TRIUMPH' => 'TRIUMPH',
'TVR' => 'TVR',
'VAUXHALL' => 'VAUXHALL',
'VESPA' => 'VESPA',
'VICTORY' => 'VICTORY',
'VOLKSWAGEN' => 'VOLKSWAGEN',
'VOLVO' => 'VOLVO',
'VOXAN' => 'VOXAN',
'WIESMANN' => 'WIESMANN',
'YAMAHA' => 'YAMAHA',
'YCF' => 'YCF',
'ZERO' => 'ZERO',
'ZONGSHEN' => 'ZONGSHEN'
)
),
'model' => array(
'name' => 'Modèle',
'type' => 'text',
'title' => 'Get the exact name on LaCentrale'
),
'versions' => array(
'name' => 'Version(s)',
'type' => 'text',
'title' => 'Get the exact name(s) on LaCentrale. Separate by comma'
),
'category' => array(
'name' => 'Catégorie',
'type' => 'list',
'values' => array(
'' => '',
'Voiture' => array(
'4x4, SUV & Crossover' => '47',
'Citadine' => '40',
'Berline' => '41_42',
'Break' => '43',
'Cabriolet' => '46',
'Coupé' => '45',
'Monospace' => '44',
'Bus et minibus' => '82',
'Fourgonnette' => '85',
'Fourgon (< 3,5 tonnes)' => '81',
'Pick-up' => '50',
'Voiture société, commerciale' => '80',
'Sans permis' => '48',
'Camion (> 3,5 tonnes)' => '83',
),
'Camion/Pickup' => array(
'Camion (> 3,5 tonnes)' => '83',
'Fourgon (< 3,5 tonnes)' => '81',
'Bus et minibus' => '82',
'Fourgonnette' => '85',
'Pick-up' => '50',
'Voiture société, commerciale' => '80'
),
'Moto' => array(
'Custom' => '60',
'Offroad' => '61',
'Roadster' => '62',
'GT' => '63',
'Mini moto' => '64',
'Mobylette' => '65',
'Supermotard' => '66',
'Trail' => '67',
'Side-car' => '69',
'Sportive' => '68'
),
'Caravane/Camping-car' => array(
'Caravane' => '423',
'Profilé' => '506',
'Fourgon aménagé' => '507',
'Intégral' => '508',
'Capucine' => '510'
)
)
),
'pricemin' => array(
'name' => 'Prix min',
'type' => 'number'
),
'pricemax' => array(
'name' => 'Prix max',
'type' => 'number'
),
'location' => array(
'name' => 'CP ou département',
'type' => 'number',
'title' => 'Only one'
),
'distance' => array(
'name' => 'Rayon de recherche',
'type' => 'list',
'values' => array(
'' => '',
'10 km' => '1',
'20 km' => '2',
'50 km' => '3',
'100 km' => '4',
'200 km' => '5'
)
),
'region' => array(
'name' => 'Région',
'type' => 'list',
'values' => array(
'' => '',
'Auvergne-Rhône-Alpes' => 'FR-ARA',
'Bourgogne-Franche-Comté' => 'FR-BFC',
'Bretagne' => 'FR-BRE',
'Centre-Val de Loire' => 'FR-CVL',
'Corse' => 'FR-COR',
'Grand Est' => 'FR-GES',
'Hauts-de-France' => 'FR-HDF',
'Île-de-France' => 'FR-IDF',
'Normandie' => 'FR-NOR',
'Nouvelle-Aquitaine' => 'FR-PAC',
'Occitanie' => 'FR-PDL',
'Pays de la Loire' => 'FR-OCC',
'Provence-Alpes-Côte d\'Azur' => 'FR-NAQ'
)
),
'mileagemin' => array(
'name' => 'Kilométrage min',
'type' => 'number'
),
'mileagemax' => array(
'name' => 'Kilométrage max',
'type' => 'number'
),
'yearmin' => array(
'name' => 'Année min',
'type' => 'number'
),
'yearmax' => array(
'name' => 'Année max',
'type' => 'number'
),
'cubiccapacitymin' => array(
'name' => 'Cylindrée min',
'type' => 'number'
),
'cubiccapacitymax' => array(
'name' => 'Cylindrée max',
'type' => 'number'
),
'fuel' => array(
'name' => 'Énergie',
'type' => 'list',
'values' => array(
'' => '',
'Diesel' => 'dies',
'Essence' => 'ess',
'Électrique' => 'elec',
'Hybride' => 'hyb',
'GPL' => 'gpl',
'Bioéthanol' => 'eth',
'Autre' => 'alt'
)
),
'gearbox' => array(
'name' => 'Boite de vitesse',
'type' => 'list',
'values' => array(
'' => '',
'Boite automatique' => 'AUTO',
'Boite mécanique' => 'MANUAL'
)
),
'doors' => array(
'name' => 'Nombre de portes',
'type' => 'list',
'values' => array(
'' => '',
'2 portes' => '2',
'3 portes' => '3',
'4 portes' => '4',
'5 portes' => '5',
'6 portes ou plus' => '6'
)
),
'firsthand' => array(
'name' => 'Première main',
'type' => 'checkbox'
),
'seller' => array(
'name' => 'Vendeur',
'type' => 'list',
'values' => array(
'' => '',
'Particulier' => 'PART',
'Professionel' => 'PRO'
)
),
'sort' => array(
'name' => 'Tri',
'type' => 'list',
'values' => array(
'Prix (croissant)' => 'priceAsc',
'Prix (décroissant)' => 'priceDesc',
'Marque (croissant)' => 'makeAsc',
'Marque (décroissant)' => 'makeDesc',
'Kilométrage (croissant)' => 'mileageAsc',
'Kilométrage (décroissant)' => 'mileageDesc',
'Année (croissant)' => 'yearAsc',
'Année (décroissant)' => 'yearDesc',
'Département (croissant)' => 'visitPlaceAsc',
'Département (décroissant)' => 'visitPlaceDesc'
)
),
));
public function collectData(){
// check data
if(!empty($this->getInput('distance'))
&& is_null($this->getInput('location'))) {
returnClientError('You need a place ("CP ou département") to search arround.');
}
$params = array(
'vertical' => $this->getInput('type'),
'makesModelsCommercialNames' => $this->getInput('brand') . ':' . $this->getInput('model'),
'versions' => $this->getInput('versions'),
'categories' => $this->getInput('category'),
'priceMin' => $this->getInput('pricemin'),
'priceMax' => $this->getInput('pricemax'),
'dptCp' => $this->getInput('location'),
'distance' => $this->getInput('distance'),
'regions' => $this->getInput('region'),
'mileageMin' => $this->getInput('mileagemin'),
'mileageMax' => $this->getInput('mileagemax'),
'yearMin' => $this->getInput('yearmin'),
'yearMax' => $this->getInput('yearmax'),
'cubicMin' => $this->getInput('cubiccapacitymin'),
'cubicMax' => $this->getInput('cubiccapacitymax'),
'energies' => $this->getInput('fuel'),
'firstHand' => $this->getInput('firsthand') ? 'true' : 'false',
'gearbox' => $this->getInput('gearbox'),
'doors' => $this->getInput('doors'),
'sortBy' => $this->getInput('sort')
);
$url = self::URI . 'listing?' . http_build_query($params);
$html = getSimpleHTMLDOM($url)
or returnServerError('Could not request LaCentrale.');
foreach($html->find('.linkAd') as $element) {
$item = array();
$item['uri'] = trim(self::URI, '/') . $element->href;
$item['title'] = $element->find('.brandModel', 0)->plaintext;
$item['sellerType'] = $element->find('.typeSeller', 0)->plaintext;
$item['author'] = $item['sellerType'];
$item['version'] = $element->find('.version', 0)->plaintext;
$item['price'] = $element->find('.fieldPrice', 0)->plaintext;
$item['year'] = $element->find('.fieldYear', 0)->plaintext;
$item['mileage'] = $element->find('.fieldMileage', 0)->plaintext;
$item['departement'] = str_replace(',', '', $element->find('.dptCont', 0)->plaintext);
$item['thumbnail'] = $element->find('.imgContent img', 0)->src;
$item['enclosures'] = array($item['thumbnail']);
$item['content'] = '
<img src="' . $item['thumbnail'] . '">
<br>Variation : ' . $item['version']
. '<br>Prix : ' . $item['price']
. '<br>Année : ' . $item['year']
. '<br>Kilométrage : ' . $item['mileage']
. '<br>Département : ' . $item['departement']
. '<br>Type de vendeur : ' . $item['sellerType'];
$this->items[] = $item;
}
}
}

View File

@ -356,6 +356,7 @@ class LeBonCoinBridge extends BridgeAbstract {
$data = $this->buildRequestJson();
$header = array(
'User-Agent: LBC;Android;Null;Null;Null;Null;Null;Null;Null;Null',
'Content-Type: application/json',
'Content-Length: ' . strlen($data),
'api_key: ' . self::$LBC_API_KEY
@ -430,11 +431,11 @@ class LeBonCoinBridge extends BridgeAbstract {
);
if($this->getInput('region') != '') {
$requestJson->filters->location['regions'] = [$this->getInput('region')];
$requestJson->filters->location['regions'] = array($this->getInput('region'));
}
if($this->getInput('department') != '') {
$requestJson->filters->location['departments'] = [$this->getInput('department')];
$requestJson->filters->location['departments'] = array($this->getInput('department'));
}
if($this->getInput('cities') != '') {
@ -466,7 +467,7 @@ class LeBonCoinBridge extends BridgeAbstract {
}
if($this->getInput('estate') != '') {
$requestJson->filters->enums['real_estate_type'] = [$this->getInput('estate')];
$requestJson->filters->enums['real_estate_type'] = array($this->getInput('estate'));
}
if($this->getInput('roomsmin') != ''
@ -525,7 +526,7 @@ class LeBonCoinBridge extends BridgeAbstract {
}
if($this->getInput('fuel') != '') {
$requestJson->filters->enums['fuel'] = [$this->getInput('fuel')];
$requestJson->filters->enums['fuel'] = array($this->getInput('fuel'));
}
$requestJson->limit = 30;

View File

@ -0,0 +1,22 @@
<?php
class ListverseBridge extends FeedExpander {
const MAINTAINER = 'IceWreck';
const NAME = 'Listverse Bridge';
const URI = 'https://listverse.com/';
const CACHE_TIMEOUT = 3600;
const DESCRIPTION = 'RSS feed for Listverse';
public function collectData(){
$this->collectExpandableDatas('https://listverse.com/feed/', 15);
}
protected function parseItem($newsItem){
$item = parent::parseItem($newsItem);
// $articlePage gets the entire page's contents
$articlePage = getSimpleHTMLDOM($newsItem->link);
$article = $articlePage->find('#articlecontentonly', 0);
$item['content'] = $article;
return $item;
}
}

View File

@ -3,7 +3,7 @@ class MangareaderBridge extends BridgeAbstract {
const MAINTAINER = 'logmanoriginal';
const NAME = 'Mangareader Bridge';
const URI = 'http://www.mangareader.net';
const URI = 'https://www.mangareader.net';
const CACHE_TIMEOUT = 10800; // 3h
const DESCRIPTION = 'Returns the latest updates, popular mangas or manga updates (new chapters)';

View File

@ -0,0 +1,89 @@
<?php
class MastodonBridge extends FeedExpander {
const MAINTAINER = 'husim0';
const NAME = 'Mastodon Bridge';
const CACHE_TIMEOUT = 900; // 15mn
const DESCRIPTION = 'Returns toots';
const URI = 'https://mastodon.social';
const PARAMETERS = array(array(
'canusername' => array(
'name' => 'Canonical username (ex : @sebsauvage@framapiaf.org)',
'required' => true,
),
'norep' => array(
'name' => 'Without replies',
'type' => 'checkbox',
'title' => 'Only return initial toots'
),
'noboost' => array(
'name' => 'Without boosts',
'required' => false,
'type' => 'checkbox',
'title' => 'Hide boosts'
)
));
public function getName(){
switch($this->queriedContext) {
case 'By username':
return $this->getInput('canusername');
default: return parent::getName();
}
}
protected function parseItem($newItem){
$item = parent::parseItem($newItem);
$content = str_get_html($item['content']);
$title = str_get_html($item['title']);
$item['title'] = $content->plaintext;
if(strlen($item['title']) > 75) {
$item['title'] = substr($item['title'], 0, strpos(wordwrap($item['title'], 75), "\n")) . '...';
}
if(strpos($title, 'shared a status by') !== false) {
if($this->getInput('noboost')) {
return null;
}
preg_match('/shared a status by (\S{0,})/', $title, $matches);
$item['title'] = 'Boost ' . $matches[1] . ' ' . $item['title'];
$item['author'] = $matches[1];
} else {
$item['author'] = $this->getInput('canusername');
}
// Check if it's a initial toot or a response
if($this->getInput('norep') && preg_match('/^@.+/', trim($content->plaintext))) {
return null;
}
return $item;
}
private function getInstance(){
preg_match('/^@[a-zA-Z0-9_]+@(.+)/', $this->getInput('canusername'), $matches);
return $matches[1];
}
private function getUsername(){
preg_match('/^@([a-zA-Z_0-9_]+)@.+/', $this->getInput('canusername'), $matches);
return $matches[1];
}
public function getURI(){
if($this->getInput('canusername'))
return 'https://' . $this->getInstance() . '/users/' . $this->getUsername() . '.atom';
return parent::getURI();
}
public function collectData(){
return $this->collectExpandableDatas($this->getURI());
}
}

View File

@ -30,29 +30,34 @@ class MediapartBridge extends FeedExpander {
protected function parseItem($newsItem) {
$item = parent::parseItem($newsItem);
// Enable single page mode?
if ($this->getInput('single_page_mode') === true) {
$item['uri'] .= '?onglet=full';
}
// Mediapart provide multiple type of contents.
// We only process items relative to the newspaper
// See issue #1292 - https://github.com/RSS-Bridge/rss-bridge/issues/1292
if (strpos($item['uri'], self::URI . 'journal/') === 0) {
// Enable single page mode?
if ($this->getInput('single_page_mode') === true) {
$item['uri'] .= '?onglet=full';
}
// If a session cookie is defined, get the full article
$mpsessid = $this->getInput('mpsessid');
if (!empty($mpsessid)) {
// Set the session cookie
$opt = array();
$opt[CURLOPT_COOKIE] = 'MPSESSID=' . $mpsessid;
// If a session cookie is defined, get the full article
$mpsessid = $this->getInput('mpsessid');
if (!empty($mpsessid)) {
// Set the session cookie
$opt = array();
$opt[CURLOPT_COOKIE] = 'MPSESSID=' . $mpsessid;
// Get the page
$articlePage = getSimpleHTMLDOM(
$newsItem->link . '?onglet=full',
array(),
$opt);
// Get the page
$articlePage = getSimpleHTMLDOM(
$newsItem->link . '?onglet=full',
array(),
$opt);
// Extract the article content
$content = $articlePage->find('div.content-article', 0)->innertext;
$content = sanitize($content);
$content = defaultLinkTo($content, static::URI);
$item['content'] .= $content;
// Extract the article content
$content = $articlePage->find('div.content-article', 0)->innertext;
$content = sanitize($content);
$content = defaultLinkTo($content, static::URI);
$item['content'] .= $content;
}
}
return $item;

View File

@ -15,11 +15,11 @@ class N26Bridge extends BridgeAbstract
public function collectData()
{
$html = getSimpleHTMLDOM(self::URI . '/en-fr/blog-archive')
$html = getSimpleHTMLDOM(self::URI . '/en-eu/blog-archive')
or returnServerError('Error while downloading the website content');
foreach($html->find('div.ga') as $article) {
$item = [];
foreach($html->find('div[class="ag ah ai aj bs bt dx ea fo gx ie if ih ii ij ik s"]') as $article) {
$item = array();
$item['uri'] = self::URI . $article->find('h2 a', 0)->href;
$item['title'] = $article->find('h2 a', 0)->plaintext;
@ -27,9 +27,9 @@ class N26Bridge extends BridgeAbstract
$fullArticle = getSimpleHTMLDOM($item['uri'])
or returnServerError('Error while downloading the full article');
$dateElement = $fullArticle->find('span[class="fk fl de ch fm by"]', 0);
$dateElement = $fullArticle->find('time', 0);
$item['timestamp'] = strtotime($dateElement->plaintext);
$item['content'] = $fullArticle->find('main article', 0)->innertext;
$item['content'] = $fullArticle->find('div[class="af ag ah ai an"]', 1)->innertext;
$this->items[] = $item;
}

60
bridges/NFLRUSBridge.php Normal file
View File

@ -0,0 +1,60 @@
<?php
class NFLRUSBridge extends BridgeAbstract {
const NAME = 'NFLRUS';
const URI = 'http://nflrus.ru/';
const DESCRIPTION = 'Returns the recent articles published on nflrus.ru';
const MAINTAINER = 'Maxim Shpak';
private function getEnglishMonth($month) {
$months = array(
'Января' => 'January',
'Февраля' => 'February',
'Марта' => 'March',
'Апреля' => 'April',
'Мая' => 'May',
'Июня' => 'June',
'Июля' => 'July',
'Августа' => 'August',
'Сентября' => 'September',
'Октября' => 'October',
'Ноября' => 'November',
'Декабря' => 'December',
);
if (isset($months[$month])) {
return $months[$month];
}
return false;
}
private function extractArticleTimestamp($article) {
$time = $article->find('time', 0);
if($time) {
$timestring = trim($time->plaintext);
$parts = explode(' ', $timestring);
$month = $this->getEnglishMonth($parts[1]);
if ($month) {
$timestring = $parts[0] . ' ' . $month . ' ' . $parts[2];
return strtotime($timestring);
}
}
return 0;
}
public function collectData() {
$html = getSimpleHTMLDOM(self::URI)
or returnServerError('Unable to get any articles from NFLRUS');
$html = defaultLinkTo($html, self::URI);
foreach($html->find('article') as $article) {
$item = array();
$item['uri'] = $article->find('.b-article__title a', 0)->href;
$item['title'] = $article->find('.b-article__title a', 0)->plaintext;
$item['author'] = $article->find('.link-author', 0)->plaintext;
$item['timestamp'] = $this->extractArticleTimestamp($article);
$item['content'] = $article->find('div', 0)->innertext;
$this->items[] = $item;
}
}
}

26
bridges/NYTBridge.php Normal file
View File

@ -0,0 +1,26 @@
<?php
class NYTBridge extends FeedExpander {
const MAINTAINER = 'IceWreck';
const NAME = 'New York Times Bridge';
const URI = 'https://www.nytimes.com/';
const CACHE_TIMEOUT = 3600;
const DESCRIPTION = 'RSS feed for the New York Times';
public function collectData(){
$this->collectExpandableDatas('https://rss.nytimes.com/services/xml/rss/nyt/HomePage.xml', 15);
}
protected function parseItem($newsItem){
$item = parent::parseItem($newsItem);
// $articlePage gets the entire page's contents
$articlePage = getSimpleHTMLDOM($newsItem->link);
// figure contain's the main article image
$article = $articlePage->find('figure', 0);
// p > css-exrw3m has the actual article
foreach($articlePage->find('p.css-exrw3m') as $element)
$article = $article . $element;
$item['content'] = $article;
return $item;
}
}

View File

@ -0,0 +1,194 @@
<?php
class NationalGeographicBridge extends BridgeAbstract {
const CONTEXT_BY_TOPIC = 'By Topic';
const PARAMETER_TOPIC = 'topic';
const PARAMETER_FULL_ARTICLE = 'full';
const TOPIC_MAGAZINE = 'Magazine';
const TOPIC_LATEST_STORIES = 'Latest Stories';
const NAME = 'National Geographic';
const URI = 'https://www.nationalgeographic.com/';
const DESCRIPTION = 'Fetches the latest articles from the National Geographic Magazine';
const MAINTAINER = 'logmanoriginal';
const PARAMETERS = array(
self::CONTEXT_BY_TOPIC => array(
self::PARAMETER_TOPIC => array(
'name' => 'Topic',
'type' => 'list',
'values' => array(
self::TOPIC_MAGAZINE => 'magazine',
self::TOPIC_LATEST_STORIES => 'latest-stories'
),
'title' => 'Select your topic',
'defaultValue' => 'Magazine'
)
),
'global' => array(
self::PARAMETER_FULL_ARTICLE => array(
'name' => 'Full Article',
'type' => 'checkbox',
'title' => 'Enable to load full articles (takes longer)'
)
)
);
private $topicName = '';
public function getURI() {
switch ($this->queriedContext) {
case self::CONTEXT_BY_TOPIC: {
return self::URI . $this->getInput(self::PARAMETER_TOPIC);
} break;
default: {
return parent::getURI();
}
}
}
public function collectData() {
$this->topicName = $this->getTopicName($this->getInput(self::PARAMETER_TOPIC));
switch($this->topicName) {
case self::TOPIC_MAGAZINE: {
return $this->collectMagazine();
} break;
case self::TOPIC_LATEST_STORIES: {
return $this->collectLatestStories();
} break;
default: {
returnServerError('Unknown topic: "' . $this->topicName . '"');
}
}
}
public function getName() {
switch ($this->queriedContext) {
case self::CONTEXT_BY_TOPIC: {
return static::NAME . ': ' . $this->topicName;
} break;
default: {
return parent::getName();
}
}
}
private function getTopicName($topic) {
return array_search($topic, static::PARAMETERS[self::CONTEXT_BY_TOPIC][self::PARAMETER_TOPIC]['values']);
}
private function collectMagazine() {
$uri = $this->getURI();
$html = getSimpleHTMLDOM($uri)
or returnServerError('Could not request ' . $uri);
$script = $html->find('#lead-component script')[0];
$json = json_decode($script->innertext, true);
// This is probably going to break in the future, fix it then :)
foreach($json['body']['0']['multilayout_promo_beta']['stories'] as $story) {
$this->addStory($story);
}
}
private function collectLatestStories() {
$uri = self::URI . 'latest-stories/_jcr_content/content/hubfeed.promo-hub-feed-all-stories.json';
$json_raw = getContents($uri)
or returnServerError('Could not request ' . $uri);
foreach(json_decode($json_raw, true) as $story) {
$this->addStory($story);
}
}
private function addStory($story) {
$title = 'Unknown title';
$content = '';
foreach($story['components'] as $component) {
switch($component['content_type']) {
case 'title': {
$title = $component['title']['text'];
} break;
case 'dek': {
$content = $component['dek']['text'];
} break;
}
}
$item = array();
$item['uri'] = $story['uri'];
$item['title'] = $title;
// if full article is requested!
if ($this->getInput(self::PARAMETER_FULL_ARTICLE))
$item['content'] = $this->getFullArticle($item['uri']);
else
$item['content'] = $content;
if (isset($story['promo_image'])) {
switch($story['promo_image']['content_type']) {
case 'image': {
$item['enclosures'][] = $story['promo_image']['image']['uri'];
} break;
}
}
if (isset($story['lead_media'])) {
$media = $story['lead_media'];
switch($media['content_type']) {
case 'image': {
// Don't add if promo_image was added
if (empty($item['enclosures']))
$item['enclosures'][] = $media['image']['uri'];
} break;
case 'image_gallery': {
foreach($media['image_gallery']['images'] as $image) {
$item['enclosures'][] = $image['uri'];
}
} break;
}
}
$this->items[] = $item;
}
private function getFullArticle($uri) {
$html = getSimpleHTMLDOMCached($uri)
or returnServerError('Could not load ' . $uri);
$html = defaultLinkTo($html, $uri);
$content = '';
foreach($html->find('
.content > .smartbody.text,
.content > .section.image script[type="text/json"],
.content > .section.image span[itemprop="caption"],
.content > .section.inline script[type="text/json"]
') as $element) {
if ($element->tag === 'script') {
$json = json_decode($element->innertext, true);
if (isset($json['src'])) {
$content .= '<img src="' . $json['src'] . '" width="100%" alt="' . $json['alt'] . '">';
} elseif (isset($json['galleryType']) && isset($json['endpoint'])) {
$doc = getContents($json['endpoint'])
or returnServerError('Could not load ' . $json['endpoint']);
$json = json_decode($doc, true);
foreach($json['items'] as $item) {
$content .= '<p>' . $item['caption'] . '</p>';
$content .= '<img src="' . $item['url'] . '" width="100%" alt="' . $item['caption'] . '">';
}
}
} else {
$content .= $element->outertext;
}
}
return $content;
}
}

View File

@ -3,7 +3,7 @@ class NiceMatinBridge extends FeedExpander {
const MAINTAINER = 'pit-fgfjiudghdf';
const NAME = 'NiceMatin';
const URI = 'http://www.nicematin.com/';
const URI = 'https://www.nicematin.com/';
const DESCRIPTION = 'Returns the 10 newest posts from NiceMatin (full text)';
public function collectData(){

View File

@ -17,6 +17,15 @@ class NineGagBridge extends BridgeAbstract {
'Fresh' => 'fresh',
),
),
'video' => array(
'name' => 'Filter Video',
'type' => 'list',
'values' => array(
'NotFiltred' => 'none',
'VideoFiltred' => 'without',
'VideoOnly' => 'only',
),
),
'p' => array(
'name' => 'Pages',
'type' => 'number',
@ -121,13 +130,32 @@ class NineGagBridge extends BridgeAbstract {
}
foreach ($posts as $post) {
$item['uri'] = $post['url'];
$item['title'] = $post['title'];
$item['content'] = self::getContent($post);
$item['categories'] = self::getCategories($post);
$item['timestamp'] = self::getTimestamp($post);
$AvoidElement = false;
switch ($this->getInput('video')) {
case 'without':
if ($post['type'] === 'Animated') {
$AvoidElement = true;
}
break;
case 'only':
echo $post['type'];
if ($post['type'] !== 'Animated') {
$AvoidElement = true;
}
break;
case 'none': default:
break;
}
$this->items[] = $item;
if (!$AvoidElement) {
$item['uri'] = $post['url'];
$item['title'] = $post['title'];
$item['content'] = self::getContent($post);
$item['categories'] = self::getCategories($post);
$item['timestamp'] = self::getTimestamp($post);
$this->items[] = $item;
}
}
}

View File

@ -3,7 +3,7 @@ class NovelUpdatesBridge extends BridgeAbstract {
const MAINTAINER = 'albirew';
const NAME = 'Novel Updates';
const URI = 'http://www.novelupdates.com/';
const URI = 'https://www.novelupdates.com/';
const CACHE_TIMEOUT = 21600; // 6h
const DESCRIPTION = 'Returns releases from Novel Updates';
const PARAMETERS = array( array(

View File

@ -1,9 +1,9 @@
<?php
class WhydBridge extends BridgeAbstract {
class OpenwhydBridge extends BridgeAbstract {
const MAINTAINER = 'kranack';
const NAME = 'Whyd Bridge';
const URI = 'http://www.whyd.com/';
const NAME = 'Openwhyd Bridge';
const URI = 'https://openwhyd.org';
const CACHE_TIMEOUT = 600; // 10min
const DESCRIPTION = 'Returns 10 newest music from user profile';
@ -17,8 +17,7 @@ class WhydBridge extends BridgeAbstract {
private $userName = '';
public function getIcon() {
return self::URI . 'assets/favicons/
32-6b62a9f14d5e1a9213090d8f00f286bba3a6022381a76390d1d0926493b12593.png?v=6';
return self::URI . '/images/favicon.ico';
}
public function collectData(){
@ -26,11 +25,11 @@ class WhydBridge extends BridgeAbstract {
if(strlen(preg_replace('/[^0-9a-f]/', '', $this->getInput('u'))) == 24) {
// is input the userid ?
$html = getSimpleHTMLDOM(
self::URI . 'u/' . preg_replace('/[^0-9a-f]/', '', $this->getInput('u'))
self::URI . '/u/' . preg_replace('/[^0-9a-f]/', '', $this->getInput('u'))
) or returnServerError('No results for this query.');
} else { // input may be the username
$html = getSimpleHTMLDOM(
self::URI . 'search?q=' . urlencode($this->getInput('u'))
self::URI . '/search?q=' . urlencode($this->getInput('u'))
) or returnServerError('No results for this query.');
for($j = 0; $j < 5; $j++) {
@ -57,6 +56,6 @@ class WhydBridge extends BridgeAbstract {
}
public function getName(){
return (!empty($this->userName) ? $this->userName . ' - ' : '') . 'Whyd Bridge';
return (!empty($this->userName) ? $this->userName . ' - ' : '') . 'Openwhyd Bridge';
}
}

View File

@ -3,7 +3,7 @@ class ParuVenduImmoBridge extends BridgeAbstract {
const MAINTAINER = 'polo2ro';
const NAME = 'Paru Vendu Immobilier';
const URI = 'http://www.paruvendu.fr';
const URI = 'https://www.paruvendu.fr';
const CACHE_TIMEOUT = 10800; // 3h
const DESCRIPTION = 'Returns the ads from the first page of search result.';

203
bridges/PatreonBridge.php Normal file
View File

@ -0,0 +1,203 @@
<?php
class PatreonBridge extends BridgeAbstract {
const NAME = 'Patreon Bridge';
const URI = 'https://www.patreon.com/';
const CACHE_TIMEOUT = 300; // 5min
const DESCRIPTION = 'Returns posts by creators on Patreon';
const MAINTAINER = 'Roliga';
const PARAMETERS = array( array(
'creator' => array(
'name' => 'Creator',
'type' => 'text',
'required' => true,
'title' => 'Creator name as seen in their page URL'
)
));
public function collectData(){
$html = getSimpleHTMLDOMCached($this->getURI(), 86400)
or returnServerError('Failed to load creator page at ' . $this->getURI());
$regex = '#/api/campaigns/([0-9]+)#';
if(preg_match($regex, $html->save(), $matches) > 0) {
$campaign_id = $matches[1];
} else {
returnServerError('Could not find campaign ID');
}
$query = array(
'include' => implode(',', array(
'user',
'attachments',
'user_defined_tags',
//'campaign',
//'poll.choices',
//'poll.current_user_responses.user',
//'poll.current_user_responses.choice',
//'poll.current_user_responses.poll',
//'access_rules.tier.null',
//'images.null',
//'audio.null'
)),
'fields' => array(
'post' => implode(',', array(
//'change_visibility_at',
//'comment_count',
'content',
//'current_user_can_delete',
//'current_user_can_view',
//'current_user_has_liked',
//'embed',
'image',
//'is_paid',
//'like_count',
//'min_cents_pledged_to_view',
//'patreon_url',
//'patron_count',
//'pledge_url',
//'post_file',
//'post_metadata',
//'post_type',
'published_at',
'teaser_text',
//'thumbnail_url',
'title',
//'upgrade_url',
'url',
//'was_posted_by_campaign_owner'
)),
'user' => implode(',', array(
//'image_url',
'full_name',
//'url'
))
),
'filter' => array(
'contains_exclusive_posts' => true,
'is_draft' => false,
'campaign_id' => $campaign_id
),
'sort' => '-published_at'
);
$posts = $this->apiGet('posts', $query);
foreach($posts->data as $post) {
$item = array(
'uri' => $post->attributes->url,
'title' => $post->attributes->title,
'timestamp' => $post->attributes->published_at,
'content' => '',
'uid' => 'patreon.com/' . $post->id
);
$user = $this->findInclude($posts,
'user',
$post->relationships->user->data->id);
$item['author'] = $user->full_name;
if(isset($post->attributes->image))
$item['content'] .= '<p><a href="'
. $post->attributes->url
. '"><img src="'
. $post->attributes->image->thumb_url
. '" /></a></p>';
if(isset($post->attributes->content)) {
$item['content'] .= $post->attributes->content;
} elseif (isset($post->attributes->teaser_text)) {
$item['content'] .= '<p>'
. $post->attributes->teaser_text
. '</p>';
}
if(isset($post->relationships->user_defined_tags)) {
$item['categories'] = array();
foreach($post->relationships->user_defined_tags->data as $tag) {
$attrs = $this->findInclude($posts, 'post_tag', $tag->id);
$item['categories'][] = $attrs->value;
}
}
if(isset($post->relationships->attachments)) {
$item['enclosures'] = array();
foreach($post->relationships->attachments->data as $attachment) {
$attrs = $this->findInclude($posts, 'attachment', $attachment->id);
$item['enclosures'][] = $attrs->url;
}
}
$this->items[] = $item;
}
}
/*
* Searches the "included" array in an API response and returns attributes
* for the first match.
*/
private function findInclude($data, $type, $id) {
foreach($data->included as $include)
if($include->type === $type && $include->id === $id)
return $include->attributes;
}
private function apiGet($endpoint, $query_data = array()) {
$query_data['json-api-version'] = 1.0;
$query_data['json-api-use-default-includes'] = 0;
$url = 'https://www.patreon.com/api/'
. $endpoint
. '?'
. http_build_query($query_data);
/*
* Accept-Language header and the CURL cipher list are for bypassing the
* Cloudflare anti-bot protection on the Patreon API. If this ever breaks,
* here are some other project that also deal with this:
* https://github.com/mikf/gallery-dl/issues/342
* https://github.com/daemionfox/patreon-feed/issues/7
* https://www.patreondevelopers.com/t/api-returning-cloudflare-challenge/2025
* https://github.com/splitbrain/patreon-rss/issues/4
*/
$header = array(
'Accept-Language: en-US',
'Content-Type: application/json'
);
$opts = array(
CURLOPT_SSL_CIPHER_LIST => implode(':', array(
'DEFAULT',
'!DHE-RSA-CHACHA20-POLY1305'
))
);
$data = json_decode(getContents($url, $header, $opts))
or returnServerError('API request to "' . $url . '" failed.');
return $data;
}
public function getName(){
if(!is_null($this->getInput('creator')))
return $this->getInput('creator') . ' posts';
return parent::getName();
}
public function getURI(){
if(!is_null($this->getInput('creator')))
return self::URI . $this->getInput('creator');
return parent::getURI();
}
public function detectParameters($url){
$params = array();
// Matches e.g. https://www.patreon.com/SomeCreator
$regex = '/^(https?:\/\/)?(www\.)?patreon\.com\/([^\/&?\n]+)/';
if(preg_match($regex, $url, $matches) > 0) {
$params['creator'] = urldecode($matches[3]);
return $params;
}
return null;
}
}

View File

@ -3,7 +3,7 @@ class PickyWallpapersBridge extends BridgeAbstract {
const MAINTAINER = 'nel50n';
const NAME = 'PickyWallpapers Bridge';
const URI = 'http://www.pickywallpapers.com/';
const URI = 'https://www.pickywallpapers.com/';
const CACHE_TIMEOUT = 43200; // 12h
const DESCRIPTION = 'Returns the latests wallpapers from PickyWallpapers';

View File

@ -32,6 +32,13 @@ class PikabuBridge extends BridgeAbstract {
'required' => true
),
'filter' => self::PARAMETERS_FILTER
),
'По пользователю' => array(
'user' => array(
'name' => 'Пользователь',
'exampleValue' => 'admin',
'required' => true
)
)
);
@ -40,6 +47,8 @@ class PikabuBridge extends BridgeAbstract {
public function getURI() {
if ($this->getInput('tag')) {
return self::URI . '/tag/' . rawurlencode($this->getInput('tag')) . '/' . rawurlencode($this->getInput('filter'));
} else if ($this->getInput('user')) {
return self::URI . '/@' . rawurlencode($this->getInput('user'));
} else if ($this->getInput('community')) {
$uri = self::URI . '/community/' . rawurlencode($this->getInput('community'));
if ($this->getInput('filter') != 'hot') {
@ -101,6 +110,10 @@ class PikabuBridge extends BridgeAbstract {
}
}
$img->outertext = '<img src="' . $src . '">';
// it is assumed, that img's parents are links to post itself
// we don't need them
$img->parent()->outertext = $img->outertext;
}
$categories = array();
@ -116,7 +129,10 @@ class PikabuBridge extends BridgeAbstract {
$item['categories'] = $categories;
$item['author'] = $post->find('.user__nick', 0)->innertext;
$item['title'] = $title->plaintext;
$item['content'] = strip_tags(backgroundToImg($post->find('.story__content-inner', 0)->innertext), '<br><p><img>');
$item['content'] = strip_tags(
backgroundToImg($post->find('.story__content-inner', 0)->innertext),
'<br><p><img><a>
');
$item['uri'] = $title->href;
$item['timestamp'] = strtotime($time->getAttribute('datetime'));
$this->items[] = $item;

View File

@ -16,12 +16,6 @@ class PinterestBridge extends FeedExpander {
'name' => 'board',
'required' => true
)
),
'From search' => array(
'q' => array(
'name' => 'Keyword',
'required' => true
)
)
);
@ -29,22 +23,14 @@ class PinterestBridge extends FeedExpander {
return 'https://s.pinimg.com/webapp/style/images/favicon-9f8f9adf.png';
}
public function collectData(){
switch($this->queriedContext) {
case 'By username and board':
$this->collectExpandableDatas($this->getURI() . '.rss');
$this->fixLowRes();
break;
case 'From search':
default:
$html = getSimpleHTMLDOMCached($this->getURI());
$this->getSearchResults($html);
}
public function collectData() {
$this->collectExpandableDatas($this->getURI() . '.rss');
$this->fixLowRes();
}
private function fixLowRes() {
$newitems = [];
$newitems = array();
$pattern = '/https\:\/\/i\.pinimg\.com\/[a-zA-Z0-9]*x\//';
foreach($this->items as $item) {
@ -55,71 +41,21 @@ class PinterestBridge extends FeedExpander {
}
private function getSearchResults($html){
$json = json_decode($html->find('#jsInit1', 0)->innertext, true);
$results = $json['resourceDataCache'][0]['data']['results'];
public function getURI() {
foreach($results as $result) {
$item = array();
$item['uri'] = self::URI . $result['board']['url'];
// Some use regular titles, others provide 'advanced' infos, a few
// provide even less info. Thus we attempt multiple options.
$item['title'] = trim($result['title']);
if($item['title'] === '')
$item['title'] = trim($result['rich_summary']['display_name']);
if($item['title'] === '')
$item['title'] = trim($result['grid_description']);
$item['timestamp'] = strtotime($result['created_at']);
$item['username'] = $result['pinner']['username'];
$item['fullname'] = $result['pinner']['full_name'];
$item['avatar'] = $result['pinner']['image_small_url'];
$item['author'] = $item['username'] . ' (' . $item['fullname'] . ')';
$item['content'] = '<img align="left" style="margin: 2px 4px;" src="'
. htmlentities($item['avatar'])
. '" /><p><strong>'
. $item['username']
. '</strong><br>'
. $item['fullname']
. '</p><br><img src="'
. $result['images']['736x']['url']
. '" alt="" /><br><p>'
. $result['description']
. '</p>';
$item['enclosures'] = array($result['images']['orig']['url']);
$this->items[] = $item;
if ($this->queriedContext === 'By username and board') {
return self::URI . '/' . urlencode($this->getInput('u')) . '/' . urlencode($this->getInput('b'));
}
return parent::getURI();
}
public function getURI(){
switch($this->queriedContext) {
case 'By username and board':
$uri = self::URI . '/' . urlencode($this->getInput('u')) . '/' . urlencode($this->getInput('b'));// . '.rss';
break;
case 'From search':
$uri = self::URI . '/search/?q=' . urlencode($this->getInput('q'));
break;
default: return parent::getURI();
}
return $uri;
}
public function getName() {
public function getName(){
switch($this->queriedContext) {
case 'By username and board':
$specific = $this->getInput('u') . ' - ' . $this->getInput('b');
break;
case 'From search':
$specific = $this->getInput('q');
break;
default: return parent::getName();
if ($this->queriedContext === 'By username and board') {
return $this->getInput('u') . ' - ' . $this->getInput('b') . ' - ' . self::NAME;
}
return $specific . ' - ' . self::NAME;
return parent::getName();
}
}

View File

@ -0,0 +1,88 @@
<?php
class PirateCommunityBridge extends BridgeAbstract {
const NAME = 'Pirate-Community Bridge';
const URI = 'https://raymanpc.com/';
const CACHE_TIMEOUT = 300; // 5min
const DESCRIPTION = 'Returns replies to topics';
const MAINTAINER = 'Roliga';
const PARAMETERS = array( array(
't' => array(
'name' => 'Topic ID',
'type' => 'number',
'title' => 'Topic ID from topic URL. If the URL contains t=12 the ID is 12.',
'required' => true
)));
private $feedName = '';
public function detectParameters($url){
$parsed_url = parse_url($url);
if($parsed_url['host'] !== 'raymanpc.com')
return null;
parse_str($parsed_url['query'], $parsed_query);
if($parsed_url['path'] === '/forum/viewtopic.php'
&& array_key_exists('t', $parsed_query)) {
return array('t' => $parsed_query['t']);
}
return null;
}
public function getName() {
if(!empty($this->feedName))
return $this->feedName;
return parent::getName();
}
public function getURI(){
if(!is_null($this->getInput('t'))) {
return self::URI
. 'forum/viewtopic.php?t='
. $this->getInput('t')
. '&sd=d'; // sort posts decending by ate so first page has latest posts
}
return parent::getURI();
}
public function collectData(){
$html = getSimpleHTMLDOM($this->getURI())
or returnServerError('Could not retrieve topic page at ' . $this->getURI());
$this->feedName = $html->find('head title', 0)->plaintext;
foreach($html->find('.post') as $reply) {
$item = array();
$item['uri'] = $this->getURI()
. $reply->find('h3 a', 0)->getAttribute('href');
$item['title'] = $reply->find('h3 a', 0)->plaintext;
$author_html = $reply->find('.author', 0);
// author_html contains the timestamp as text directly inside it,
// so delete all other child elements
foreach($author_html->children as $child)
$child->outertext = '';
// Timestamps are always in UTC+1
$item['timestamp'] = trim($author_html->innertext) . ' +01:00';
$item['author'] = $reply
->find('.username, .username-coloured', 0)
->plaintext;
$item['content'] = defaultLinkTo($reply->find('.content', 0)->innertext,
$this->getURI());
$item['enclosures'] = array();
foreach($reply->find('.attachbox img.postimage') as $img)
$item['enclosures'][] = urljoin($this->getURI(), $img->src);
$this->items[] = $item;
}
}
}

View File

@ -0,0 +1,67 @@
<?php
/**
* PlantUML releases bridge showing latest releases content
* @author nicolas-delsaux
*
*/
class PlantUMLReleasesBridge extends BridgeAbstract
{
const MAINTAINER = 'Riduidel';
const NAME = 'PlantUML Releases';
const AUTHOR = 'PlantUML team';
// URI is no more valid, since we can address the whole gq galaxy
const URI = 'http://plantuml.com/fr/changes';
const CACHE_TIMEOUT = 7200; // 2h
const DESCRIPTION = 'PlantUML releases bridge, showing for each release the changelog';
const DEFAULT_DOMAIN = 'plantuml.com';
const PARAMETERS = array( array(
));
const REPLACED_ATTRIBUTES = array(
'href' => 'href',
'src' => 'src',
'data-original' => 'src'
);
private function getDomain() {
$domain = $this->getInput('domain');
if (empty($domain))
$domain = self::DEFAULT_DOMAIN;
if (strpos($domain, '://') === false)
$domain = 'https://' . $domain;
return $domain;
}
public function getURI()
{
return self::URI;
}
public function collectData()
{
$html = getSimpleHTMLDOM($this->getURI()) or returnServerError('Could not request ' . $this->getURI());
// Since GQ don't want simple class scrapping, let's do it the hard way and ... discover content !
$main = $html->find('div[id=root]', 0);
foreach ($main->find('h2') as $release) {
$item = array();
$item['author'] = self::AUTHOR;
$release_text = $release->innertext;
if (preg_match('/(.+) \((.*)\)/', $release_text, $matches)) {
$item['title'] = $matches[1];
// And now, build the date from the date text
$item['timestamp'] = strtotime($matches[2]);
}
$item['uri'] = $this->getURI();
$item['content'] = $release->next_sibling ();
$this->items[] = $item;
}
}
}

View File

@ -1,44 +0,0 @@
<?php
class ReadComicsBridge extends BridgeAbstract {
const MAINTAINER = 'niawag';
const NAME = 'Read Comics';
const URI = 'http://www.readcomics.tv/';
const DESCRIPTION = 'Enter the comics as they appear in the website uri,
separated by semicolons, ex: good-comic-1;good-comic-2; ...';
const PARAMETERS = array( array(
'q' => array(
'name' => 'keywords, separated by semicolons',
'exampleValue' => 'first list;second list;...',
'required' => true
),
));
public function collectData(){
function parseDateTimestamp($element){
$guessedDate = $element->find('span', 0)->plaintext;
$guessedDate = strptime($guessedDate, '%m/%d/%Y');
$timestamp = mktime(0, 0, 0, $guessedDate['tm_mon'] + 1, $guessedDate['tm_mday'], date('Y'));
return $timestamp;
}
$keywordsList = explode(';', $this->getInput('q'));
foreach($keywordsList as $keywords) {
$html = $this->getSimpleHTMLDOM(self::URI . 'comic/' . rawurlencode($keywords))
or $this->returnServerError('Could not request readcomics.tv.');
foreach($html->find('li') as $element) {
$item = array();
$item['uri'] = $element->find('a.ch-name', 0)->href;
$item['id'] = $item['uri'];
$item['timestamp'] = parseDateTimestamp($element);
$item['title'] = $element->find('a.ch-name', 0)->plaintext;
if(isset($item['title']))
$this->items[] = $item;
}
}
}
}

40
bridges/RedditBridge.php Normal file
View File

@ -0,0 +1,40 @@
<?php
class RedditBridge extends FeedExpander {
const MAINTAINER = 'leomaradan';
const NAME = 'Reddit Bridge';
const URI = 'https://www.reddit.com/';
const DESCRIPTION = 'Reddit RSS Feed fixer';
const PARAMETERS = array(
'single' => array(
'r' => array(
'name' => 'SubReddit',
'required' => true,
'exampleValue' => 'selfhosted',
'title' => 'SubReddit name'
)
),
'multi' => array(
'rs' => array(
'name' => 'SubReddits',
'required' => true,
'exampleValue' => 'selfhosted, php',
'title' => 'SubReddit names, separated by commas'
)
)
);
public function collectData(){
switch($this->queriedContext) {
case 'single': $subreddits[] = $this->getInput('r'); break;
case 'multi': $subreddits = explode(',', $this->getInput('rs')); break;
}
foreach ($subreddits as $subreddit) {
$name = trim($subreddit);
$this->collectExpandableDatas("https://www.reddit.com/r/$name/.rss");
}
}
}

View File

@ -9,22 +9,6 @@ class Releases3DSBridge extends BridgeAbstract {
public function collectData(){
function typeToString($type){
switch($type) {
case 1: return '3DS Game';
case 4: return 'eShop';
default: return '??? (' . $type . ')';
}
}
function cardToString($card){
switch($card) {
case 1: return 'Regular (CARD1)';
case 2: return 'NAND (CARD2)';
default: return '??? (' . $card . ')';
}
}
$dataUrl = self::URI . 'xml.php';
$xml = getContents($dataUrl)
or returnServerError('Could not request 3dsdb: ' . $dataUrl);
@ -95,8 +79,8 @@ class Releases3DSBridge extends BridgeAbstract {
. '<br /><b>Release Name: </b>' . $releasename
. '<br /><b>Trimmed size: </b>' . intval(intval($trimmedsize) / 1048576)
. 'MB<br /><b>Firmware: </b>' . $firmware
. '<br /><b>Type: </b>' . typeToString($type)
. '<br /><b>Card: </b>' . cardToString($card)
. '<br /><b>Type: </b>' . $this->typeToString($type)
. '<br /><b>Card: </b>' . $this->cardToString($card)
. '<br />';
//Build search links section to facilitate release search using search engines
@ -124,4 +108,20 @@ class Releases3DSBridge extends BridgeAbstract {
$limit++;
}
}
private function typeToString($type){
switch($type) {
case 1: return '3DS Game';
case 4: return 'eShop';
default: return '??? (' . $type . ')';
}
}
private function cardToString($card){
switch($card) {
case 1: return 'Regular (CARD1)';
case 2: return 'NAND (CARD2)';
default: return '??? (' . $card . ')';
}
}
}

View File

@ -3,7 +3,7 @@ class ReporterreBridge extends BridgeAbstract {
const MAINTAINER = 'nyutag';
const NAME = 'Reporterre Bridge';
const URI = 'http://www.reporterre.net/';
const URI = 'https://www.reporterre.net/';
const DESCRIPTION = 'Returns the newest articles.';
private function extractContent($url){

View File

@ -25,7 +25,7 @@ class RoadAndTrackBridge extends BridgeAbstract {
private function fixImages($content) {
$enclosures = [];
$enclosures = array();
foreach($content->find('img') as $image) {
$image->src = explode('?', $image->getAttribute('data-src'))[0];
$enclosures[] = $image->src;

View File

@ -5,7 +5,7 @@ class Rule34Bridge extends GelbooruBridge {
const MAINTAINER = 'mitsukarenai';
const NAME = 'Rule34';
const URI = 'http://rule34.xxx/';
const URI = 'https://rule34.xxx/';
const DESCRIPTION = 'Returns images from given page';
const PIDBYPAGE = 50;

View File

@ -5,6 +5,23 @@ class Rule34pahealBridge extends Shimmie2Bridge {
const MAINTAINER = 'mitsukarenai';
const NAME = 'Rule34paheal';
const URI = 'http://rule34.paheal.net/';
const URI = 'https://rule34.paheal.net/';
const DESCRIPTION = 'Returns images from given page';
protected function getItemFromElement($element){
$item = array();
$item['uri'] = $this->getURI() . $element->href;
$item['id'] = (int)preg_replace('/[^0-9]/', '', $element->getAttribute(static::IDATTRIBUTE));
$item['timestamp'] = time();
$thumbnailUri = $element->find('img', 0)->src;
$item['tags'] = $element->getAttribute('data-tags');
$item['title'] = $this->getName() . ' | ' . $item['id'];
$item['content'] = '<a href="'
. $item['uri']
. '"><img src="'
. $thumbnailUri
. '" /></a><br>Tags: '
. $item['tags'];
return $item;
}
}

View File

@ -5,7 +5,7 @@ class SafebooruBridge extends GelbooruBridge {
const MAINTAINER = 'mitsukarenai';
const NAME = 'Safebooru';
const URI = 'http://safebooru.org/';
const URI = 'https://safebooru.org/';
const DESCRIPTION = 'Returns images from given page';
const PIDBYPAGE = 40;

View File

@ -1,11 +0,0 @@
<?php
require_once('MoebooruBridge.php');
class SakugabooruBridge extends MoebooruBridge {
const MAINTAINER = 'mitsukarenai';
const NAME = 'Sakugabooru';
const URI = 'http://sakuga.yshi.org/';
const DESCRIPTION = 'Returns images from given page';
}

View File

@ -3,7 +3,7 @@ class ScmbBridge extends BridgeAbstract {
const MAINTAINER = 'Astalaseven';
const NAME = 'Se Coucher Moins Bête Bridge';
const URI = 'http://secouchermoinsbete.fr';
const URI = 'https://secouchermoinsbete.fr';
const CACHE_TIMEOUT = 21600; // 6h
const DESCRIPTION = 'Returns the newest anecdotes.';

Some files were not shown because too many files have changed in this diff Show More