Commit graph

16 commits

Author SHA1 Message Date
Chuck Adams
80eeb6b727
Trim unused packages (#50)
* refactor: zap doctrine/orm (but still using DBAL)

* refactor: zap flysystem

* build: upgrade dev deps

* refactor: zap CacheServiceInterface and DatabaseCacheService

* refactor: zap VersionUtil
2025-02-22 16:19:59 -07:00
Chuck Adams
9bfdb8b309
switch to laravel pint, drop house style and use PER-CS (#45)
* tweak: remove our house style, leaving only PER-CS2.0 and PHP84Migration

* qa: replace php-cs-fixer with pint
2025-01-31 12:00:45 -07:00
Chuck Adams
f9b671a879
new script-based update strategy (#44)
* feat: dead simple and dead slow svn checkout scripts for plugins/themes

* feat: add scripts to check for updated plugins/themes

* tweak: split slugs on commas or whitespace instead of just commas

* tweak: replace --plugins and --themes options with --slugs and --slugs-from

* tweak: add logging config to Caddyfile

* feat: push-sync and update-sync commands

* build: bump composer versions
2025-01-28 08:27:53 -07:00
Chuck Adams
565d1d1660
use php-cs-fixer for style linting (#37)
* chore: add friendsofphp/php-cs-fixer

* chore: run make fix

* ci: add style check as first test action

* chore: use .cache dir for phpunit
2024-12-01 09:41:52 -07:00
Chuck Adams
f1d3e572de
use \Safe versions of builtin functions (#36) 2024-12-01 09:04:35 -07:00
Chuck Adams
fdc1ced7ec
convert to full Symfony framework (#35)
* feat: switch to symfony/skeleton 7.2

* chore: move AspirePress\AspireSync -> App

* refactor: rm styled output which is redundant with monolog

* docs: update README for bin/console

* refactor: rename commands to all use sync: prefix

* chore: rm bin/aspiresync

* feat: add SyncAsset and SyncResource entities

* chore: add doctrine-migrations-bindle

* feat: add migration

* feat: new docker-compose stack

* chore: switch to postgres backend
2024-12-01 08:04:14 -07:00
Chuck Adams
b6f2db28ac
File Metadata parser for eventual first-party use (+gardening) (#32)
* feat: RegexUtil::replace and _replace methods

* feat: FileMetadataParser for reading plugin metadata header

* ci: actually do ci :)

* feat: read theme headers

* refactor: rename FileMetadataParser -> FileHeaderParser

* fix: appease phpstan
2024-11-24 20:16:19 -07:00
Chuck Adams
c25cd6795f
Use Saloon to asynchronously fetch metadata (#31)
* fix: lolphp of numeric string => int conversion in addNewAndRequested

* refactor: make themes commands somewhat more generic

* refactor: rename WpEndpointClient -> DotOrgApiClient

* build: add saloonphp/saloon

* refactor: replace DotOrgApiService with saloon

* refactor: move request constructors to abstract base

* refactor: rearrange, refactor, and rename commands and services

* refactor: pull configure() method up to base class in sync commands

* feat: use async pool with concurrency of 10

* feat: use saloon for downloads

* fix: don't escape slashes in log json

* chore: comment out some excess log spam

* fix: mark 404 downloads as processed

* refactor: rename Resource -> ResourceType (Resource is just too common)

* feat: replace progressive backoff with saloonphp/rate-limit-plugin

* fix: disable HasRateLimits trait, it's severely bugged with async
2024-11-24 06:14:58 -07:00
Chuck Adams
537ed90276
Big Refactors Part 2: DBAL (#29)
* wip: replacing pdo with dbal

* wip: nuke stats, we'll use the db for that

* wip: refactoring ListService, adding caching

* refactor: rm unnecessary cache in SubversionService

* chore: drop aspiresync.sqlite in the app root

* fix: urldecode slugs scraped from svn

* feat: add RegexUtil for a more convenient regex api

* fix: add blank version lists to fetched subversion slugs

* test: wipe out all tests except the one unit test

* refactor: rename getUpdatedListOfItems -> getUpdatedItems

* refactor: rm $action param from getItemsForAction

* fix: drop action in ThemeListService::getItemsForAction

* refactor: move most ListService methods into AbstractListService

* fix: move AbstractMetadataService up one dir

* refactor: fully factor all six of plugin/theme services into base

* fix: add missing columns in saveOpen

* chore: port RevisionMetadataService to use DBAL

* chore: drop aura/sql

* fix: catch any fetch errors that slip past Guzzle

* fix: convert numeric string keys to real strings

* test: new stubs and useless unit test

* chore: put more useful data in aspirepress_meta

* chore: don't save aspirepress_meta, we can generate it during dump

* feat: create Logger service from katzgrau/KLogger

* refactor: fix IDE warnings in Logger.php, strip out doc comments

* feat: wire up log service

* feat: LogService is now JsonLogger and less than 100 LOC

* chore: only include context when non-empty

* fix: use double quotes in messages so strings interpolate

* fix: log to tempfile by default, stdout doesn't make much sense yet

* refactor: change getCategory() to a readonly property

* fix: various fixes around logging (+tests)

* fix: use static:: consistently in JsonLogger

* chore: add lots of debug trace logging

* fix: default log file to /dev/null

* feat: generate aspiresync_meta on export

* chore: add debug logging in metadata service

* fix: use constructor injection for loggers because of readonly class
2024-11-15 10:57:52 -07:00
Chuck Adams
72b4bcb0c6
de-dockerization, take two (#28) 2024-11-10 14:22:21 -07:00
Alex Sirota
f61e3cbb99
Revert "prefer local environment to docker, refactor Makefiles" 2024-11-10 04:31:26 -05:00
Chuck Adams
d5f28d3f64 feat: new composer scripts 2024-11-09 15:52:10 -07:00
Chuck Adams
f6ee19761c
Major refactor to use sqlite, symfony DI, (#19)
* refactor: fix most ide warnings

* wip: gardening

* fix: switch php container from alpine to ubuntu

* feat: add juicefs client to container

* fix: generate en_US.UTF-8 locales in container

* feat: add some scripts for working with juicefs

* fix: use juicefs installer instead of hamfisted COPY

* refactor: switch db backend to sqlite

* fix: change some sql bits to sqlite syntax

* style: clean up sqlite schema

* fix: enable foreign key pragma

* wip: save metadata directly to db for all response types

* feat: implement --skip-newer-than-secs

* feat: report closed plugins at info level so real errors stand out

* refactor: remove meta:import:plugins command

* fix: write not-found status for Plugin not found

* feat: sync theme metadata directly to sqlite

* chore: go back to the original names of plugins:meta and themes:meta

* wip: cleanups

* chore: drop UtilCleanCommand since the filesystem is no longer much used

* wip: gardening

* fix: don't send any user-agent header at all

* style: inline factory constructor args and use named args

* wip: replace laminas service manager with symfony's container

* chore: zap unneeded factories

* chore: rm postgres stuff from build, replace bootstrap with make init

* tmp: move laminas-style config into attic

* test: disable SvnServiceTest due to external network calls

* wip: continue replacing laminas-servicemanager

* refactor: use #[AutoWire] attributes in ExtendedPdoFactory

* wip: use resolveEnvPlaceHolders: true  (tho we don't use them now)

* refactor: normalize naming of commands

* chore: abort the juicefs experiment

* fix: use right default db name in makefile

* fix: append versions to outcomes array

* fix: add UA for downloads (empty UA is blocked)

* fix: add missing zip package

* refactor: use flysystem for download commands

* refactor: drop RunAllCommand, we have make for that

* chore: clear out attic (UtilUploadCommand is not coming back)

* chore: rm unused CallbackInterface.php

* refactor: rename GetItemsFromSourceTrait -> HasStats

* wip: download commands wip

* fix: change full timeout to 15 from 5

* wip: ProcessManager

* wip: single version at a time download jobs

* fix: drop silly --ansi option, that's for symfony console

* fix: use right params in getDownloadUrl sql

* chore: improve logging in plugins:download:single

* chore: run composer bump

* chore: upgrade and bump

* docs: fix typos in README.md (credit to @AmnestyAm)

* chore: rename DOWNLOAD_DIR to DOWNLOADS_DIR for consistency

* docs: update README.md to reflect the latest configuration (+reformat)

* doc: rm redundant USAGE from README.md (we already have command help)

* docs: rm LICENSE section from README.md, we'll let it speak for itself

speculation on whether or not the GPL applies definitely doesn't belong there, and we don't even use any GPL'd components I know of.

* chore: drop Partial download commands, they're largely unnecessary now

* refactor: factor Download commands into a common base

* chore: implement themes:download:single

* format: make fix

* wip: refactoring various services

* refactor: ThemesListService now uses svn like plugins

* fix: actually execute parallel processes this time

* fix: remove processStartDelay

* refactor: factor out AbstractDownloadService

* fix: disable output on process start, which suddenly stopped working

* feat: add export commands

* fix: add newlines to exports

* chore: rename export -> dump
2024-11-09 09:24:42 -07:00
Chuck Adams
ab5db6b495
expanded metadata download, test environment fixes (#16)
* feat: initialize and use a proper test environment and not the real db

* chore: rearrange status messages so slugs line up

* fix: create dirs if they don't exist

* fix: download full list of fields for theme metadata

* fix: rm request uri from retry messages that already contain it

* fix: use 1.2 api with all fields for plugin metadata

* fix: use right table name in getVersionsForUnfinalizedPlugins

* style: reformat sql into heredoc in PluginMetadataService

* feat: add FileUtil::cachedFile

* build: split make check and make fix

* feat: add symfony/var-dumper and roave/security-advisories

* chore: switch to 2 second connect timeout, 10 retries, 500ms backoff
2024-11-02 20:11:08 -06:00
Chuck Adams
8046e10daf
revision table changes, dev updates (#14)
* feat: inject Guzzle client with retry logic

* fix: replace flysystem calls with builtins when writing files

flysystem just plain falls on its face, files we tell it to write just don't show up.

* fix: write svn data to tempfile and not the final filename

* refactor: fetch only latest rows per action in revisions table

the revisions table is also now append-only: preserveRevision always inserts a new row

* feat: new FileUtil for common filesystem operations

* feat: use thecodingmachine/safe for json functions

* style: fix all files with phpcs

* fix: exempt GuzzleClientFactory from phpcs because phpcs chokes on it

* refactor: remove use of Filesystem everywhere except S3 ops
2024-10-22 19:12:35 -06:00
Chuck Adams
b5fac0e65b
skip existing downloads without -u flag, switch to standard layout (#11)
* refactor: drop code/ and build/ dirs, do everything from the root

* chore: drop unused NETWORK envar in makefile

* feat: skip downloads if metadata file already exists

* feat: add --skip-existing flag and leave -u alone
2024-10-20 13:30:33 -04:00
Renamed from code/composer.json (Browse further)