v-wordpress-plugin-updater/vendor/theseer/tokenizer/README.md
nikolai@vontainment.com afe656c55b modified: .gitignore
modified:   composer.lock
	modified:   mu-plugin/v-sys-plugin-updater-mu.php
	modified:   mu-plugin/v-sys-plugin-updater.php
	modified:   mu-plugin/v-sys-theme-updater-mu.php
	modified:   mu-plugin/v-sys-theme-updater.php
	modified:   tests/ApiKeyHelperTest.php
	modified:   update-api/app/Controllers/ApiController.php
	modified:   update-api/app/Controllers/HomeController.php
	deleted:    update-api/app/Controllers/KeyController.php
	modified:   update-api/app/Core/Router.php
	modified:   update-api/app/Models/HostsModel.php
	modified:   update-api/config.php
	modified:   update-api/public/install.php
	deleted:    update-api/storage/test.sqlite
2025-10-20 06:55:25 -04:00

47 lines
1.4 KiB
Markdown

# Tokenizer
A small library for converting tokenized PHP source code into XML.
[![Test](https://github.com/theseer/tokenizer/actions/workflows/ci.yml/badge.svg)](https://github.com/theseer/tokenizer/actions/workflows/ci.yml)
## Installation
You can add this library as a local, per-project dependency to your project using [Composer](https://getcomposer.org/):
composer require theseer/tokenizer
If you only need this library during development, for instance to run your project's test suite, then you should add it as a development-time dependency:
composer require --dev theseer/tokenizer
## Usage examples
```php
$tokenizer = new TheSeer\Tokenizer\Tokenizer();
$tokens = $tokenizer->parse(file_get_contents(__DIR__ . '/src/XMLSerializer.php'));
$serializer = new TheSeer\Tokenizer\XMLSerializer();
$xml = $serializer->toXML($tokens);
echo $xml;
```
The generated XML structure looks something like this:
```xml
<?xml version="1.0"?>
<source xmlns="https://github.com/theseer/tokenizer">
<line no="1">
<token name="T_OPEN_TAG">&lt;?php </token>
<token name="T_DECLARE">declare</token>
<token name="T_OPEN_BRACKET">(</token>
<token name="T_STRING">strict_types</token>
<token name="T_WHITESPACE"> </token>
<token name="T_EQUAL">=</token>
<token name="T_WHITESPACE"> </token>
<token name="T_LNUMBER">1</token>
<token name="T_CLOSE_BRACKET">)</token>
<token name="T_SEMICOLON">;</token>
</line>
</source>
```