Commit graph

35 commits

Author SHA1 Message Date
ORelio
47dc26c775
[NextINpact] Fix subtitle extraction in #LeBrief (#1780)
The bridge was taking another article abstract as subtitle for #LeBrief articles
2020-10-05 11:57:13 +05:00
ORelio
07c71b3b36
[NextINpact] Upgrade for NextINpact v7 (#1708) 2020-08-20 17:49:26 +05:00
ORelio
de8cee6a1c Catching up | [Main] Debug mode, parse utils, MIME | [Bridges] Add/Improve 20 bridges (#802)
* Debug mode improvements

 - Improve debug warning message
 - Restore error reporting in debug mode
 - Fix 'notice' messages for unset fields

* Add parsing utility functions

html.php
 - extractFromDelimiters
 - stripWithDelimiters
 - stripRecursiveHTMLSection
 - markdownToHtml (partial)

bridges
 - remove now-duplicate functions
 - call functions from html.php instead

* [Anidex] New bridge

Anime torrent tracker

* [Anime-Ultime] Restore thumbnail

* [CNET] Recreate bridge

Full rewrite as the previous one was broken

* [Dilbert] Minor URI fix

Use new self::URI property

* [EstCeQuonMetEnProd] Fix content extraction

Bridge was broken

* [Facebook] Fix "SpSonsSoriSsés" label

... which was taking space in item title

* [Futura-Sciences] Use HTTPS, More cleanup

Use HTTPS as FS now offer HTTPS
Clean additional useless HTML elements

* [GBATemp] Multiple fixes

- Fix categories: missing "break" statements
- Restore thumbnail as enclosure
- Fix date extraction
- Fix user blog post extraction
- Use getSimpleHTMLDOMCached

* [JapanExpo] Fix bridge, HTTPS, thumbnails

- Fix getSimpleHTMLDOMCached call
- Upgrade to HTTPS as JE now offers HTTPS
- Restore thumbnails as enclosures

* [LeMondeInformatique] Fix bridge, HTTPS

- Upgrade to HTTPS as LMI now offers HTTPS
- Restore thumbnails using small images
- Fix content extraction
- Fix text encoding issue

* [Nextgov] Fix content extraction

- Restore thumbnail and use small image
- Field extraction fixes

* [NextInpact] Add categories and filtering by type

- Offer all RSS feeds
- Allow filtering by article type
- Implement extraction for brief articles
- Remove article limit, many brief articles are publied all at once

* [NyaaTorrents] New bridge

Anime torrent tracker

* [Releases3DS] Cache content, restore thumbnail

- Use getSimpleHTMLDOMCached
- Restore thumbnail as enclosure

* [TheHackerNews] Fix bridge

 - Fix content extraction including article body
 - Restore thumbnail as enclosure

* [WeLiveSecurity] HTTPS, Fix content extraction

- Upgrade to HTTPS as WLS now offers HTTPS
- Fix content extraction including article body

* [WordPress] Reduce timeout, more content selectors

- Reduce timeout to use default one (1h)
- Add new content selector (articleBody)
- Find thumbnail and set as enclosure
- Fix <script> cleanup

* [YGGTorrent] Increase limit, use cache

- Increase item limit as uploads are very frequent
- Use getSimpleHTMLDOMCached

* [ZDNet] Rewrite with FeedExpander

- Upgrade to HTTPS as ZD now offers HTTPS
- Use FeedExpander for secondary fields
- Fix content extraction for article body

* [Main] Handle MIME type for enclosures

Many feed readers will ignore enclosures (e.g. thumbnails) with no MIME type. This commit adds automatic MIME type detection based on file extension (which may be inaccurate but is the only way without fetching the content).

One can force enclosure type using #.ext anchor (hacky, needs improving)

* [FeedExpander] Improve field extraction

- Add support for passing enclosures
- Improve author and uri extraction
- Fix 'notice' PHP error messages

* [Pull] Coding style fixes for #802

* [Pull] Implementing changes for #802

 - Fix coding style issues with str append
 - Remove useless CACHE_TIMEOUT
 - Use count() instead of $limit
 - Use defaultLinkTo() + handle strings
 - Use http_build_query()
 - Fix missing </em>
 - Remove error_reporting(0)
 - warning CSS (@LogMANOriginal)
 - Fix typo in FeedExpander comment

* [Main] More documentation for markdownToHtml

See #802 for more details
2018-09-09 20:20:13 +01:00
Corentin Garcia
2ac0469750 Updated 4 bridges to use HTTPS (#497)
* [NextInpactBridge] Use https

* [InstagramBridge] Use https

* [GBAtempBridge] Use https

* [LeBonCoinBridge] Use https
2017-03-21 20:27:12 +00:00
logmanoriginal
ff83410534 style: Fix coding styles 2017-02-14 17:28:07 +01:00
logmanoriginal
6f24858124 bridges: Fix coding styles
This commit is a squash of all commits that fix coding styles
for the new coding style policy.

[ABCTabsBridge] Fix coding style
[AcrimedBridge] Fix coding style
[AllocineFRBridge] Fix coding style
[AnimeUltimeBridge] Fix coding style
[Arte7Bridge] Fix coding style
[AskfmBridge] Fix coding style
[BandcampBridge] Fix coding style
[BastaBridge] Fix coding style
[BlaguesDeMerdeBridge] Fix coding style
[BooruprojectBridge] Fix coding style
[CADBridge] Fix coding style
[CNETBridge] Fix coding style
[CastorusBridge] Fix coding style
[CollegeDeFranceBridge] Fix coding style
[CommonDreamsBridge] Fix coding style
[CopieDoubleBridge] Fix coding style
[CourrierInternationalBridge] Fix coding style
[CpasbienBridge] Fix coding style
[CryptomeBridge] Fix coding style
[DailymotionBridge] Fix coding style
[DanbooruBridge] Fix coding style
[DansTonChatBridge] Fix coding style
[DauphineLibereBridge] Fix coding style
[DeveloppezDotComBridge] Fix coding style
[DemoBridge] Fix coding style
[DilbertBridge] Fix coding style
[DuckDuckGoBridge] Fix coding style
[DollbooruBridge] Fix coding style
[EliteDangerousGalnetBridge] Fix coding style
[ElsevierBridge] Fix coding style
[EstCeQuonMetEnProdBridge] Fix coding style
[EZTVBridge] Fix coding style
[FacebookBridge] Fix coding style
[FeedExpanderExampleBridge] Fix coding style
[FB2Bridge] Fix coding style
[FierPandaBridge] Fix coding style
[FlickrBridge] Fix coding style
[FootitoBridge] Fix coding style
[FourchanBridge] Fix coding style
[FuturaSciencesBridge] Fix coding style
[GBAtempBridge] Fix coding style
[GelbooruBridge] Fix coding style
[GiphyBridge] Fix coding style
[GithubIssueBridge] Fix coding style
[GizmodoBridge] Fix coding style
[GoComicsBridge] Fix coding style
[GooglePlusPostBridge] Fix coding style
[GoogleSearchBridge] Fix coding style
[HDWallpapersBridge] Fix coding style
[HentaiHavenBridge] Fix coding style
[IdenticaBridge] Fix coding style
[InstagramBridge] Fix coding style
[IsoHuntBridge] Fix coding style
[JapanExpoBridge] Fix coding style
[KonachanBridge] Fix coding style
[KoreusBridge] Fix coding style
[KununuBridge] Fix coding style
[LeBonCoinBridge] Fix coding style
[LegifranceJOBBridge] Fix coding style
[LeMondeInformatiqueBridge] Fix coding style
[LesJoiesDuCodeBridge] Fix coding style
[LichessBridge] Fix coding style
[LinkedInCompanyBridge] Fix coding style
[LolibooruBridge] Fix coding style
[LWNprevBridge] Fix coding style
[MangareaderBridge] Fix coding style
[MilbooruBridge] Fix coding style
[MixCloudBridge] Fix coding style
[MoebooruBridge] Fix coding style
[MondeDiploBridge] Fix coding style
[MsnMondeBridge] Fix coding style
[MspabooruBridge] Fix coding style
[NasaApodBridge] Fix coding style
[NeuviemeArtBridge] Fix coding style
[NextgovBridge] Fix coding style
[NextInpactBridge] Fix coding style
[NiceMatinBridge] Fix coding style
[NovelUpdatesBridge] Fix coding style
[OpenClassroomsBridge] Fix coding style
[ParuVenduImmoBridge] Fix coding style
[PickyWallpapersBridge] Fix coding style
[PinterestBridge] Fix coding style
[PlanetLibreBridge] Fix coding style
[ReadComicsBridge] Fix coding style
[Releases3DSBridge] Fix coding style
[ReporterreBridge] Fix coding style
[RTBFBridge] Fix coding style
[Rue89Bridge] Fix coding style
[Rule34Bridge] Fix coding style
[Rule34pahealBridge] Fix coding style
[SafebooruBridge] Fix coding style
[SakugabooruBridge] Fix coding style
[ScmbBridge] Fix coding style
[ScoopItBridge] Fix coding style
[SensCritiqueBridge] Fix coding style
[SexactuBridge] Fix coding style
[ShanaprojectBridge] Fix coding style
[Shimmie2Bridge] Fix coding style
[SoundcloudBridge] Fix coding style
[StripeAPIChangeLogBridge] Fix coding style
[SuperbWallpapersBridge] Fix coding style
[T411Bridge] Fix coding style
[TagBoardBridge] Fix coding style
[TbibBridge] Fix coding style
[TheCodingLoveBridge] Fix coding style
[TheHackerNewsBridge] Fix coding style
[ThePirateBayBridge] Fix coding style
[TheTVDBBridge] Fix coding style
[Torrent9Bridge] Fix coding style
[TwitterBridge] Fix coding style
[UnsplashBridge] Fix coding style
[ViadeoCompanyBridge] Fix coding style
[VineBridge] Fix coding style
[VkBridge] Fix coding style
[WallpaperStopBridge] Fix coding style
[WebfailBridge] Fix coding style
[WeLiveSecurityBridge] Fix coding style
[WhydBridge] Fix coding style
[WikipediaBridge] Fix coding style
[WordPressBridge] Fix coding style
[WorldOfTanksBridge] Fix coding style
[XbooruBridge] Fix coding style
[YandereBridge] Fix coding style
[YoutubeBridge] Fix coding style
[ZDNetBridge] Fix coding style
2017-02-12 15:34:08 +01:00
Pierre Mazière
f1fb95b257 [core] extract BridgeAbstract methods to make them functions
- returnError, returnServerError, returnClientError ,debugMessage are
  moved to lib/error.php

- getContents, getSimpleHTMLDOM, getSimpleHTMLDOMCached are moved to
  lib/contents.php

Signed-off-by: Pierre Mazière <pierre.maziere@gmx.com>
2016-09-25 23:22:33 +02:00
Pierre Mazière
3bacb407fe [bridges] use parent::parseItems() instead of feed type specific method
Signed-off-by: Pierre Mazière <pierre.maziere@gmx.com>
2016-09-12 10:42:27 +02:00
logmanoriginal
2eec89ab27 [bridges] Change all bridges to use BridgeAbstract with getSimpleHTMLDOMCached 2016-09-10 19:11:09 +02:00
logmanoriginal
3a94956915 [bridges] Remove calls to get_cached and remove_from_cache 2016-09-10 00:19:24 +02:00
logmanoriginal
2861a855e4 [bridges] Define max items and clear caches 2016-09-05 20:26:45 +02:00
logmanoriginal
179e73fb80 [bridges] Change to extend from FeedExpander 2016-09-05 20:10:19 +02:00
Pierre Mazière
9a0da733ef [bridges] use constants instead of variable members
Signed-off-by: Pierre Mazière <pierre.maziere@gmx.com>
2016-09-01 23:15:51 +02:00
Pierre Mazière
03cbd756a6 [NextInpactBridge] code simplification
Signed-off-by: Pierre Mazière <pierre.maziere@gmx.com>
2016-08-29 13:54:14 +02:00
Pierre Mazière
de1b39c8e5 [core + bridges] get rid of loadMetadata
if a bridge needs to modify some of the data that were initialized
there, ::__construct() should be used instead.

Signed-off-by: Pierre Mazière <pierre.maziere@gmx.com>
2016-08-28 13:05:03 +02:00
Pierre Mazière
117031bf0f [core] store parameters values in BridgeAbstract::parameters
This way, any BridgeAbstract method can now have access to these values,
no only collectData

Signed-off-by: Pierre Mazière <pierre.maziere@gmx.com>
2016-08-25 01:24:53 +02:00
logmanoriginal
bf0a9d754e [bridges] Change all occurrences of the Item object to array 2016-08-22 18:55:59 +02:00
Pierre Mazière
8e32142ce5 [bridges] fix typo in getSimpleHTMLDOM call
Signed-off-by: Pierre Mazière <pierre.maziere@gmx.com>
2016-08-21 19:48:23 +02:00
Pierre Mazière
bba216073c [bridges] remove useless 'update' property from all bridges
That's the source code manager job to keep this information
consistent

Signed-off-by: Pierre Mazière <pierre.maziere@gmx.com>
2016-08-21 00:21:41 +02:00
Pierre Mazière
3c0d13c1bb bridges: use BridgeAbstract::getSimpleHTMLDOM
instead of BridgeAbstract::file_get_html

Signed-off-by: Pierre Mazière <pierre.maziere@gmx.com>
2016-08-19 10:20:27 +02:00
logmanoriginal
74f0572d91 bridges: Replace returnError function with more specific
Replacements depend on original error code:
400: returnClientError
404: returnServerError
500: returnServerError
501: returnServerError
2016-08-17 14:45:08 +02:00
logmanoriginal
63bb7ef24d bridges: Remove unnecessary functions getName() and getURI() 2016-08-09 20:08:46 +02:00
logmanoriginal
42b6c82753 bridges: Cleanup occurrences of $item->thumbnailUri
This is a subsequent change to f3eefab
2016-08-09 15:50:55 +02:00
logmanoriginal
5ad157d2fd bridges: Remove getCacheDuration if default
BridgeAbstract will return 3600 seconds by default, so the function
can be removed from any bridge implementing getCacheDuration
returning the same value.

Documentation updated accordingly.
2016-08-06 16:23:09 +02:00
logmanoriginal
1d53b70272 bridges: Remove unused functions and update scopes
Many bridges implemented getDescription() which is only applicable for
bridges extending RSSExpander.

Functions that are not part of the abstract class or interface should
be in private scope for better readability.
2016-08-06 16:00:56 +02:00
logmanoriginal
b889b51a9e Fix indentation and remove empty lines 2016-08-03 21:26:53 +02:00
logmanoriginal
58322137d3 Fix author tag in source ('author' -> 'creator') 2016-08-03 21:26:00 +02:00
logmanoriginal
1afdf9cef7 Change all nested functions to member functions
This fixes error "Using $this when not in object context"

Nested functions are not part of the object and therefore don't have
access to the object instance $this!
2016-08-03 21:19:00 +02:00
Pierre Mazière
955eecc299 use BridgeAbstract::file_get_html in all bridges
instead of simple_html_dom function file_get_html

Signed-off-by: Pierre Mazière <pierre.maziere@gmx.com>
2016-06-25 23:17:42 +02:00
teromene
3cb652b812 Deuxième lot de bridges. 2015-12-06 15:56:39 +01:00
ORelio
c8ef31bac6 [NextInpact] Fix php notice message
For non-premium articles, a notice is generated because of trying to get
property from a non-object. Fixed that with a proper check using is_object
2015-10-23 12:19:12 +02:00
ORelio
8b9c40534b NextInpactBridge: Add mention for premium articles
Add mention on article bottom (n% to discover)
Minor name fix: Nextinpact -> NextInpact
Further code indentation fixes
2015-09-05 14:42:55 +02:00
ORelio
80fe041f32 NextImpactBridge Improvements
- Add subtitle
- Add news main image
- Add thumbnail rss field
- Add author rss field
- Minor code cleanup
2015-09-04 18:11:04 +02:00
Paul Vayssiere
94a6b42a2b Add the funny text that completes the title 2014-07-14 13:39:41 -05:00
Mitsukarenai
3b558e1c2a Add extra bridges, first batch (So feed, Many RSS) 2014-05-26 00:30:46 +02:00