- Ruby 58.5%
- JavaScript 35.5%
- HTML 3.1%
- SCSS 2.8%
7e30fa8196
This is the last of the upload-tooling PRs, and it deletes the one setting I could never pick a good default for: thread_count_factor. The benchmarks (mt/upload-bench-harness) made the problem obvious. For the exact same code, the best worker count runs from about 8 (local store, CPU-bound image cooking) up to about 24 (an 8-core box pushing to S3, where every worker just parks on PUT latency). No single factor is right in both places, so instead of guessing I let the pipeline find the level while it runs and adjust as it goes. Throughput is the primary signal, not CPU. On the local-store image path the box tops out at only 27-47% CPU while throughput has already flatlined, because the real ceiling is the single writer thread plus the GVL-serialized Ruby per upload, not the cores. So "CPU still has headroom" does not mean more workers will help. The controller watches items/s and, when two increases in a row each buy less than ~5%, it reverts the last step and holds for a while before probing again. That plateau guard is what actually catches the ceiling, whatever the cause (writer thread, GVL, subprocess saturation, S3 bandwidth). CPU stays as a guard rail: over ~95% it backs off, then waits a couple of ticks because the in-flight convert subprocesses lag the signal. Memory comes first, ahead of everything, because an OOM kills the run and a bit of oscillation does not. When available memory drops near empty the controller halves the target (allowed below the normal floor) and freezes increases for a few ticks; when it is merely low it just blocks increases for that tick. It reads both /proc/meminfo and the cgroup v2 limit and takes the tighter of the two, so a container capped well under the host RAM is respected. As a second layer, since one pathological convert can balloon within a single 2s tick, each task also sets MAGICK_MEMORY_LIMIT/MAGICK_MAP_LIMIT to a sane bound at startup (unless the operator set one, and never above what policy.xml allows). The existing max_image_megapixels caps the decode size on top of that. Pieces: - WorkerGate: an adjustable semaphore. The pipeline spawns ceiling-many worker threads once; a worker takes a permit around each item, so raising or lowering the target is just workers waking or parking at the next item boundary, no threads killed or respawned. The permit is taken outside with_connection, so a parked worker never pins an AR connection. - ResourceSampler: CPU busy from /proc/stat (which sees the ImageMagick children and Postgres, not just our threads), with a Process.times fallback where procfs is missing; memory headroom from meminfo + cgroup v2. cgroup v1 is deliberately skipped (noted in the code) since the tooling only runs on v2 or bare metal. - AdaptiveController: the ~2s control loop, with the seed and hard bounds in one place. Seed is today's heuristic (usable_cpus * 1.5 * store_factor) so it starts familiar; ceiling is min(ar_pool - 8, (external ? 16 : 4) * usable_cpus, fd_headroom), and it raises the NOFILE soft limit to the hard one at startup. I moved WorkerBudget.usable_cpus to a shared Migrations::SystemInfo, since the converter and the uploads pipeline now both need it. Not built, on purpose: a PID controller (overkill for a signal this noisy), loadavg (the 1-minute EMA is far too laggy for 2s ticks), and per-worker blocked-state introspection. |
||
|---|---|---|
| .agents | ||
| .claude | ||
| .cursor/rules | ||
| .devcontainer | ||
| .github | ||
| .skills | ||
| .vscode | ||
| .zed | ||
| app | ||
| bin | ||
| config | ||
| db | ||
| docs | ||
| frontend | ||
| images | ||
| lib | ||
| log | ||
| migrations | ||
| patches | ||
| plugins | ||
| public | ||
| script | ||
| spec | ||
| stylelint-rules | ||
| test | ||
| themes | ||
| vendor | ||
| .annotaterb.yml | ||
| .editorconfig | ||
| .git-blame-ignore-revs | ||
| .gitattributes | ||
| .gitignore | ||
| .ignore | ||
| .jsdoc | ||
| .licensed.yml | ||
| .licensee.json | ||
| .npmrc | ||
| .pnpmfile.cjs | ||
| .prettierignore | ||
| .prettierrc.cjs | ||
| .rspec | ||
| .rspec_parallel | ||
| .rubocop.yml | ||
| .ruby-gemset.sample | ||
| .streerc | ||
| AGENTS.md | ||
| AI-AGENTS.md | ||
| Brewfile | ||
| CLAUDE.md | ||
| CODEOWNERS | ||
| config.ru | ||
| CONTRIBUTING.md | ||
| COPYRIGHT.md | ||
| d | ||
| discourse.sublime-project | ||
| eslint.config.mjs | ||
| Gemfile | ||
| Gemfile.lock | ||
| GEMINI.md | ||
| lefthook.yml | ||
| LICENSE.txt | ||
| package.json | ||
| pnpm-lock.yaml | ||
| pnpm-workspace.yaml | ||
| Rakefile | ||
| README.md | ||
| stylelint.config.mjs | ||
| translator.yml | ||
| tsconfig-base.json | ||
| tsconfig.json | ||
| versions.json | ||
The online home for your community.
You can self-host Discourse on your own infrastructure. But if you'd rather skip the setup, maintenance, and server management, we offer official Discourse hosting.
👉 Learn more about Discourse hosting
Discourse is a 100% open-source community platform for those who want complete control over how and where their site is run.
Our platform has been battle-tested for over a decade and continues to evolve to meet users’ needs for a powerful community platform.
With Discourse, you can:
-
💬 Create discussion topics to foster meaningful conversations.
-
⚡️ Connect in real-time with built-in chat.
-
🎨 Customize your experience with an ever-growing selection of official and community themes.
-
🤖 Enhance your community with plugins, from chatbots powered by Discourse AI to advanced tools like SQL analysis with the Data Explorer plugin.
To learn more, visit discourse.org and join our support community at meta.discourse.org.
Here are just a few of the incredible communities using Discourse:
👉 Discover more communities using Discourse
Development
To get your environment set up, follow one of the setup guides:
- Docker
- Dev Container in VS Code (recommended)
- CLI
- macOS
- Ubuntu/Debian
- Windows
Before you get started, ensure you have the following minimum versions: Ruby 3.4+, PostgreSQL 15, Redis 7.
For more information, check out the Developer Documentation.
Setting up Discourse
If you want to set up a Discourse forum for production use, see our Discourse Install Guide.
If you're looking for official hosting, see discourse.org/pricing.
Requirements
Discourse supports the latest, stable releases of all major browsers and platforms:
| Browsers | Tablets | Phones |
|---|---|---|
| Apple Safari | iPadOS | iOS |
| Google Chrome | Android | Android |
| Microsoft Edge | ||
| Mozilla Firefox |
Additionally, we aim to support Safari on iOS 16.4+.
Built With
- Ruby on Rails — Our back end API is a Rails app. It responds to requests RESTfully in JSON.
- Ember.js — Our front end is an Ember.js app that communicates with the Rails API.
- PostgreSQL — Our main data store is in Postgres.
- Redis — We use Redis as a cache and for transient data.
- BrowserStack — We use BrowserStack to test on real devices and browsers.
Plus lots of Ruby Gems, a complete list of which is at /main/Gemfile.
Contributing
Discourse is 100% free and open source. We encourage and support an active, healthy community that accepts contributions from the public – including you!
Before contributing to Discourse:
- Please read the complete mission statements on discourse.org. Yes we actually believe this stuff; you should too.
- Read and sign the Electronic Discourse Forums Contribution License Agreement.
- Dig into CONTRIBUTING.MD, which covers submitting bugs, requesting new features, preparing your code for a pull request, etc.
- Always strive to collaborate with mutual respect.
- Not sure what to work on? We've got some ideas.
We look forward to seeing your pull requests!
Security
We take security very seriously at Discourse; all our code is 100% open source and peer reviewed. Please read our security guide for an overview of security measures in Discourse, or if you wish to report a security issue.
Security fixes are listed in the release notes for each version.
The Discourse Team
The original Discourse code contributors can be found in AUTHORS.MD. For a complete list of the many individuals that contributed to the design and implementation of Discourse, please refer to the official Discourse blog and GitHub's list of contributors.
Copyright / License
Copyright 2014 - 2025 Civilized Discourse Construction Kit, Inc.
Licensed under the GNU General Public License Version 2.0 (or later); you may not use this work except in compliance with the License. You may obtain a copy of the License in the LICENSE file, or at:
https://www.gnu.org/licenses/old-licenses/gpl-2.0.txt
Unless required by applicable law or agreed to in writing, software distributed under the License is distributed on an "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the License for the specific language governing permissions and limitations under the License.
Discourse logo and “Discourse Forum” ®, Civilized Discourse Construction Kit, Inc.
Accessibility
To guide our ongoing effort to build accessible software we follow the W3C’s Web Content Accessibility Guidelines (WCAG). If you'd like to report an accessibility issue that makes it difficult for you to use Discourse, email accessibility@discourse.org. For more information visit discourse.org/accessibility.
Dedication
Discourse is built with love, Internet style.
For over a decade, our amazing community has helped shape Discourse into what it is today. Your support, feedback, and contributions have been invaluable in making Discourse a powerful and versatile platform.
We’re deeply grateful for every feature request, bug report, and discussion that has driven Discourse forward. Thank you for being a part of this journey—we couldn’t have done it without you!
