2
0
Fork 0
mirror of https://github.com/discourse/discourse.git synced 2025-10-03 17:21:20 +08:00
Commit graph

179 commits

Author SHA1 Message Date
Jarek Radosz
52e872961a
DEV: Clean up requires (#34946)
1. remove unnecessary `.rb` filename suffixes from `require_relative`
calls
2. replace `require File.expand_path(File.dirname(__FILE__) + …` with
`require_relative`
3. remove `./` prefixes from `require_relative` calls
2025-09-24 12:00:22 +02:00
Natalie Tay
a9b1fb9b43
FIX: Add ld+json VideoObject to each lazy video (#34678)
Some checks are pending
Licenses / run (push) Waiting to run
Linting / run (push) Waiting to run
Publish Assets / publish-assets (push) Waiting to run
Tests / core backend (push) Waiting to run
Tests / plugins backend (push) Waiting to run
Tests / core frontend (Chrome) (push) Waiting to run
Tests / plugins frontend (push) Waiting to run
Tests / themes frontend (push) Waiting to run
Tests / core system (push) Waiting to run
Tests / plugins system (push) Waiting to run
Tests / themes system (push) Waiting to run
Tests / core frontend (Firefox ESR) (push) Waiting to run
Tests / core frontend (Firefox Evergreen) (push) Waiting to run
Tests / chat system (push) Waiting to run
Tests / merge (push) Blocked by required conditions
Lazily loaded videos currently won't be picked up by search engines as
they do not contain an iframe or VideoObject.

This commit uses a new API `build_plugin_html
"server:topic-show-crawler-post-end"` to insert populated
[ld+json](https://developers.google.com/search/docs/appearance/structured-data/intro-structured-data#supported-formats)
into the end of each post which has a video.

Results:
-
https://search.google.com/test/rich-results/result?id=vegb8Qkz7HThsMbFaeN2AQ
-
https://search.google.com/test/rich-results/result?id=asumX41RneM2tMstchv45A
- (vimeo)
https://search.google.com/test/rich-results/result?id=EZWOMJCkDqEQIHfOLkSkcw

Note: 

There were originally two ways around this 
- add each VideoObject to the `post.cook` all the time agnostic of
crawler view
- or add them to the `<head>` with appropriate metadata, needing to
duplicate all post meta data within a `isPartOf` attribute.

I chose to create a new api so that we can satisfy both "only add the
meta data in crawler view" and prevent post metadata duplication.
2025-09-03 22:01:14 +08:00
Kris
5ddd04fa0b
UX: improve quote title alignment, wrapping (#33796)
Reported here:
https://meta.discourse.org/t/expand-quote-arrow-new-line-creates-gap-in-quote-style/375067

In cases of long content and narrow viewports, sometimes quote content
can break the layout:

<img width="244" alt="image"
src="https://github.com/user-attachments/assets/3674cabb-e278-4bec-a9cf-48623dfdce6c"
/>


This improves the situation by adding an additional wrapper to the quote
onebox title and category. This allows text to wrap independently from
the avatar and controls, and in worst cases the text truncates:

<img width="500" alt="image"
src="https://github.com/user-attachments/assets/3e29bf52-0b36-4f70-8237-22ee3b9377b2"
/>

<img width="300" alt="image"
src="https://github.com/user-attachments/assets/8fbd83bf-dafd-42cc-a470-a928c252c183"
/>

Even without a post HTML rebuild for the quotes to get the new wrapper,
the new CSS prevents layout breakage through `text-overflow: ellipsis;`
alone

Old quote without the new wrapper, but with the new CSS:

<img width="350" alt="image"
src="https://github.com/user-attachments/assets/8922bce0-17a7-4731-a1de-33daad9d7b37"
/>
2025-07-23 16:21:55 -04:00
Kris
cb0b8351a9
A11Y: onebox avatars/images are decorative, so screenreaders can skip (#33572)
Adding `alt=""` to tell screenreaders to skip these, we don't reliably
have alt content available for onebox images and they tend to be
decorative, so better to omit.


<img width="738" height="151" alt="learn.microsoft.com onebox with the
link whats new in C# 11, there's a purple C# badge to the left of the
link"
src="https://github.com/user-attachments/assets/15bdc830-683f-4c69-a11b-9d54c66fa627"
/>


We already do this in some oneboxes, so this covers some more cases
including the generic.
2025-07-11 09:55:48 -04:00
Arpit Jalan
3ff96d6cfc
FIX: correctly unescape title for amazon oneboxes (#33010)
Fixes `&amp;` in onebox title and replaces with `&`.
2025-06-02 17:50:13 +05:30
David Taylor
387dc8c255
DEV: Drop 'cache_onebox_response_body' feature (#32035)
This is a hidden site setting which has never been publicized, and is
not recommended for use. If we decide to add a feature like this in
future as a visible site setting, it would need many more safeguards to
prevent misuse.
2025-03-28 10:55:29 +00:00
Arkshine
dc9269eee5
FEATURE: Handle youtube live url format (#31673)
Meta:
https://meta.discourse.org/t/youtube-autoembeds-for-live-streams/350920

This PR supports YouTube live URLs, such as:
`https://www.youtube.com/live/eJemwqO0SDw`.


![image](https://github.com/user-attachments/assets/b7e57857-5676-4dcf-862e-1e4b4e594009)

![image](https://github.com/user-attachments/assets/f5e6f2d0-a158-41c6-bc20-0642868dbef8)
2025-03-06 15:43:38 -03:00
Gary Pendergast
8615fc6cbb
DEV: Add a user agent to all HTTP requests that Discourse makes. (#31555)
This change standardises the `User-Agent` header that Discourse will send when talking to other sites.

`Discourse.user_agent` is now the authority on what the user agent value should be. For Onebox requests, this changes the user agent from their existing value to match the new value (unless overridden).

For all other requests, `Net::HTTPHeader` is monkey-patched to add a default `User-Agent` header when one hasn't been provided.
2025-03-03 16:32:25 +11:00
Martin Brennan
ed1543455d
FIX: Allow oneboxes with no description (#31518)
This behaviour was allowed in
cb82dce86a
but then inexplicably removed a few months later in
https://github.com/discourse/onebox/pull/448, but showing
title-only oneboxes is valid. The original Meta topic that
this was discussed in was
https://meta.discourse.org/t/abc-news-not-oneboxing-due-to-missing-description/155933
.

This commit re-introduces allowing this behaviour to avoid the need for
a plugin,
c.f. https://meta.discourse.org/t/allow-title-only-onebox/354306

For example
<https://en-americas-support.nintendo.com/app/answers/detail/a_id/67660>

This commit also unhides onebox descriptions in chat, it's not
clear why they were ever hidden in the first place
2025-02-26 13:16:51 +10:00
Martin Brennan
a0e1a12161
FIX: Wikipedia onebox images and sections (#31384)
Both article images and section extraction were
not working for wikipedia oneboxes, this commit
fixes both and updates our spec fixture responses
to use the new HTML
2025-02-19 14:29:17 +10:00
David Taylor
f5c2a4dbbd
DEV: Drop experimental enable_diffhtml_preview setting (#31306)
This was intended to provide a better UX for interactive elements in the
composer preview. However, the morphing strategy has irreconcilable
conflicts with our `decorateCooked` API, and so we have been unable to
enable this by default.

Going forward, we're focussing efforts on the WYSIWYG composer to
provide this kind of smooth UX, so we're dropping the
`enable_diffhtml_preview` approach.
2025-02-12 15:58:30 +00:00
Blake Erickson
17116c440b
SECURITY: Restrict allowed URL patterns
Restrict allowed URL patterns for oneboxes.
2025-02-04 13:32:34 -03:00
Loïc Guitaut
5055a071b8 FIX: Allow to follow non-ASCII canonical links for oneboxes 2025-02-04 15:40:23 +01:00
Martin Brennan
d28a4edd1f
FIX: Hacker News onebox favicon (#31115)
This file has changed at some point to an svg instead
of a gif
2025-02-03 09:17:13 +10:00
Jarek Radosz
3a93a61092
FEATURE: Remove unnecessary org names from PR oneboxes (#31102)
Instead of displaying:

`discourse:main <- discourse:feature` or `discourse:main <-
user:feature`

shows:

`main <- feature` or `main <- user:feature`
2025-02-01 01:05:08 +01:00
Rafael dos Santos Silva
68dde7887a
FEATURE: Handle newer url format for Twitch clips (#31080) 2025-01-31 10:42:46 -03:00
David Taylor
ce97e51b63
UX: Improve git blob oneboxes (#30694)
Rearranges the info at the top of github/gitlab 'blob' oneboxes to make them easier to read
2025-01-10 11:49:36 +00:00
Gary Pendergast
f53c734ba6
FEATURE: Add a onebox_locale site setting. (#30655)
Following on from f369db5ae9, this change adds the ability to choose a custom locale to send to onebox providers.

If this setting is left blank, it will fall back to using default_locale.
2025-01-09 14:11:37 +11:00
Gary Pendergast
f369db5ae9
FIX: Ensure Onebox requests ask for the correct language. (#30637)
Onebox embeds currently default to accepting any language response from the destination, which can have some surprising behaviour. For example the `curl` equivalent of what Onebox does:

```
% curl -si -H "Accept-Language: *" 'https://developer.android.com/studio' | grep location:
location: /studio?hl=hi
```

This PR uses the value of `SiteSetting.default_locale` to populate the `Accept-Language` header, falling back to English if that isn't available, then finally accepting whatever language the destination makes available.
2025-01-09 09:08:27 +11:00
Kris
60826162b5
A11Y: remove redundant alt text from github oneboxes (#30083) 2024-12-04 12:25:03 +11:00
David Taylor
b47ae6d437
UX: Strip multiline comments in github oneboxes (#30040)
We were already stripping comments from GitHub issue/PR oneboxes, but the regex was not correctly matching multiline comments.
2024-12-02 18:08:55 +00:00
Alan Guo Xiang Tan
57f4176b57
DEV: Bump rubocop_discourse (#29608) 2024-11-06 06:27:49 +08:00
Alan Guo Xiang Tan
322a3be2db
DEV: Remove logical OR assignment of constants (#29201)
Constants should always be only assigned once. The logical OR assignment
of a constant is a relic of the past before we used zeitwerk for
autoloading and had bugs where a file could be loaded twice resulting in
constant redefinition warnings.
2024-10-16 10:09:07 +08:00
David Taylor
7d8974d02f
UX: Upgrade to fontawesome 6.6.0 (#28778)
This upgrade is designed to be fully backwards-compatible. Any icon names which have changed will be automatically remapped to the new name. For now, this will happen silently. In future, once core & official themes/plugins have been updated, we will start raising deprecation errors to help theme/plugin authors update their code.

Extracted from https://github.com/discourse/discourse/pull/28715

Announcement at https://meta.discourse.org/t/were-upgrading-our-icons-to-font-awesome-6/325349

Co-authored-by: awesomerobot <kris.aubuchon@discourse.org>
2024-09-09 14:40:56 +01:00
Natalie Tay
188cb58daa
SECURITY: Fixes for main (#28137)
* SECURITY: Update default allowed iframes list

Change the default iframe url list to all include 3 slashes.

* SECURITY: limit group tag's name length

Limit the size of a group tag's name to 100 characters.

Internal ref - t/130059

* SECURITY: Improve sanitization of SVGs in Onebox

---------

Co-authored-by: Blake Erickson <o.blakeerickson@gmail.com>
Co-authored-by: Régis Hanol <regis@hanol.fr>
Co-authored-by: David Taylor <david@taylorhq.com>
2024-07-30 14:19:01 +08:00
Vinoth Kannan
d681decf01
FEATURE: use new site setting for onebox custom user agent. (#28045)
Previously, we couldn't change the user agent name dynamically for onebox requests. In this commit, a new hidden site setting `onebox_user_agent` is created to override the default user agent value specified in the [initializer](c333e9d6e6/config/initializers/100-onebox_options.rb (L15)).

Co-authored-by: Régis Hanol <regis@hanol.fr>
2024-07-24 04:45:30 +05:30
Martin Brennan
f5cbc3e3b8
FEATURE: Allow oneboxing private GitHub repo URLs and add private indicator to HTML (#27947)
Followup 560e8aff75

The linked commit allowed oneboxing private GitHub PRs,
issues, commits, and so on, but it didn't actually allow
oneboxing the root repo e.g https://github.com/discourse/discourse-reactions

We didn't have an engine for this, we were relying on OpenGraph
tags on the HTML rendering of the page like we do with other
oneboxes.

To fix this, we needed a new github engine for repos specifically.

Also, this commit adds a `data-github-private-repo` attribute to
PR, issue, and repo onebox HTML so we have an indicator of
whether the repo was private, which can be used for theme components
and so on.
2024-07-19 12:21:45 +10:00
Martin Brennan
97e2b353f6
FEATURE: Allow for multiple GitHub onebox tokens (#27887)
Followup 560e8aff75

GitHub auth tokens cannot be made with permissions to
access multiple organisations. This is quite limiting.
This commit changes the site setting to be a "secret list"
type, which allows for a key/value mapping where the value
is treated like a password in the UI.

Now when a GitHub URL is requested for oneboxing, the
org name from the URL is used to determine which token
to use for the request.

Just in case anyone used the old site setting already,
there is a migration to create a `default` entry
with that token in the new list setting, and for
a period of time we will consider that token valid to
use for all GitHub oneboxes as well.
2024-07-15 13:07:36 +10:00
Martin Brennan
560e8aff75
FEATURE: Allow oneboxing private GitHub URLs (#27705)
This commit adds the ability to onebox private GitHub
commits, pull requests, issues, blobs, and actions using
a new `github_onebox_access_token` site setting. The token
must be set up in correctly to have access to the repos needed.

To do this successfully with the Oneboxer, we need to skip
redirects on the github.com host, otherwise we get a 404
on the URL before it is translated into a GitHub API URL
and has the appropriate headers added.
2024-07-10 09:39:31 +10:00
Alan Guo Xiang Tan
10afe5fcf1
SECURITY: Avoid the use of Object#send in Onebox::Engine::StandardEmbed
Use `Object#public_send` instead which is much safer
2024-07-03 20:49:25 +08:00
Jan Cernik
311b737c91
SECURITY: Fix Stored-dom XSS via Facebook Oneboxes 2024-07-03 20:49:22 +08:00
Rafael dos Santos Silva
b2a9676f0b
FEATURE: Support Spotify Onebox (#27540) 2024-06-19 13:27:27 -03:00
Loïc Guitaut
2a28cda15c DEV: Update to lastest rubocop-discourse 2024-05-27 18:06:14 +02:00
Jan Cernik
1ed1a1f96a
FIX: Add onebox support for x.com (#27140) 2024-05-23 10:25:42 -03:00
dsims
e6e3eaf472
FIX: avoid error from missing meta tags (#26927) 2024-05-14 11:41:53 -04:00
Alan Guo Xiang Tan
c8da2a33e8
FIX: Attempt to onebox even if response body exceeds max_download_kb (#26929)
In 95a82d608d, we lowered the default for
`Onebox.options.max_download_kb` from 10mb to 2mb for security hardening
purposes. However, this resulted in multiple bug reports where seemingly
nomral URLs stopped being oneboxed. It turns out that lowering
`Onebox.options.max_download_kb` resulted in `Onebox::Helpers::DownloadTooLarge` being raised
more often for more URLs  in `Onebox::Helpers.fetch_response` which
`Onebox::Helpers.fetch_html_doc` relies on. When
`Onebox::Helpers::DownloadTooLarge` is raised in
`Onebox::Helpers.fetch_response`, we throw away whatever response body
which we have already downloaded at that point. This is not ideal
because Nokogiri can parse incomplete HTML documents and there is a
really high chance that the incomplete HTML document still contains the
information which we need for oneboxing.

Therefore, this commit updates `Onebox::Helpers.fetch_html_doc` to not
throw away the response body when the size of the response body exceeds
`Onebox.options.max_download_size`. Instead, we just take whatever
response which we have and get Nokogiri to parse it.
2024-05-09 07:00:34 +08:00
Blake Erickson
40b707a690
FEATURE: Add onebox for loom (#26016)
Loom share links will now onebox and use the embedded loom player.
2024-03-04 15:12:08 -07:00
Roman Rizzi
a709b7e861
FIX: Allow sanitized-HTML in GH issues and categories oneboxes. (#25374)
Follow-up to d78357917c

Related meta topic: https://meta.discourse.org/t/html-is-not-render-on-category-onebox-description/289424:
2024-01-22 15:25:29 -03:00
Jan Cernik
d401502834
FIX: Localize text in github oneboxes (#25327) 2024-01-19 11:26:06 -03:00
Kris
e8509ddf50
FIX: github onebox styles for commits (#25098) 2024-01-02 17:29:56 -05:00
Jarek Radosz
694b5f108b
DEV: Fix various rubocop lints (#24749)
These (21 + 3 from previous PRs) are soon to be enabled in rubocop-discourse:

Capybara/VisibilityMatcher
Lint/DeprecatedOpenSSLConstant
Lint/DisjunctiveAssignmentInConstructor
Lint/EmptyConditionalBody
Lint/EmptyEnsure
Lint/LiteralInInterpolation
Lint/NonLocalExitFromIterator
Lint/ParenthesesAsGroupedExpression
Lint/RedundantCopDisableDirective
Lint/RedundantRequireStatement
Lint/RedundantSafeNavigation
Lint/RedundantStringCoercion
Lint/RedundantWithIndex
Lint/RedundantWithObject
Lint/SafeNavigationChain
Lint/SafeNavigationConsistency
Lint/SelfAssignment
Lint/UnreachableCode
Lint/UselessMethodDefinition
Lint/Void

Previous PRs:
Lint/ShadowedArgument
Lint/DuplicateMethods
Lint/BooleanSymbol
RSpec/SpecFilePathSuffix
2023-12-06 23:25:00 +01:00
Jarek Radosz
4280c01153
DEV: Fix Lint/ShadowedArgument (#24733) 2023-12-06 13:16:10 +01:00
Kris
797da5870b
FEATURE: remove category badge style options, set bullet style as default (#24198) 2023-11-13 10:46:15 -05:00
Ted Johansson
95a82d608d SECURITY: Prevent Onebox cache overflow by limiting downloads and URL lengths 2023-11-09 13:39:18 +11:00
Roman Rizzi
d78357917c SECURITY: Onebox templates' HTML injections.
The use of triple-curlies on Mustache templates opens the possibility for HTML injections.
2023-11-09 13:39:11 +11:00
Ted Johansson
b2a5f5802a
DEV: Replace custom Onebox symbolize_keys implementation with ActiveSupport (#23828)
We have a custom implementation of #symbolize_keys in our Onebox helpers. This is likely a legacy from when Onebox was a standalone gem. This change replaces all usages with either #deep_symbolize_keys from ActiveSupport, or appropriate option to the JSON parser gem used.
2023-10-09 09:32:09 +02:00
Ted Johansson
60e624e768
DEV: Replace custom Onebox blank implementation with ActiveSupport (#23827)
We have a custom implementation of #blank? in our Onebox helpers. This is likely a legacy from when Onebox was a standalone gem. This change replaces all usages with respective incarnations of #blank?, #present?, and #presence from ActiveSupport. It changes a bunch of "unless blank" to "if present" as well.
2023-10-07 19:54:26 +02:00
Rafael dos Santos Silva
d10e9a6c1d
FEATURE: Onebox and Download for WEBP and AVIF (#23235)
This adds support for oneboxing WEBP and AVIF images in posts and fixing
oneboxing fixes download remote images for those formats too.

Reported in https://meta.discourse.org/t/-/276433?u=falco
2023-08-24 16:44:06 -03:00
Jarek Radosz
94649565ce
DEV: Correct Style/RedundantReturn rubocop issues (#23052) 2023-08-10 02:03:38 +02:00
Joffrey JAFFEUX
df7dab9dce
FIX: ensures generic onebox has width/height for thumbnail (#23040)
Prior to this fix we would output an image with no width/height which would then bypass a large part of `CookedProcessorMixin` and have no aspect ratio. As a result, an image with no size would cause layout shift.

It also removes a fix for oneboxes in chat messages due to this case.
2023-08-09 20:31:11 +02:00