YogeLiu/TexTeller
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity

Update README_zh.md

Browse Source
This commit is contained in:
OleehyO
2024-03-25 16:35:34 +08:00
committed by GitHub
commit 14b637cd6b
60 changed files with 1792 additions and 0 deletions
Download Patch File Download Diff File Expand all files Collapse all files

7
.gitignore vendored Normal file
View File

@@ -0,0 +1,7 @@
**/__pycache__
**/.vscode
**/train_result
**/logs
**/.cache
**/tmp*

21
LICENSE Normal file
View File

@@ -0,0 +1,21 @@
MIT License
Copyright (c) 2024 OleehyO
Permission is hereby granted, free of charge, to any person obtaining a copy
of this software and associated documentation files (the "Software"), to deal
in the Software without restriction, including without limitation the rights
to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
copies of the Software, and to permit persons to whom the Software is
furnished to do so, subject to the following conditions:
The above copyright notice and this permission notice shall be included in all
copies or substantial portions of the Software.
THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE
SOFTWARE.

172
README.md Normal file
View File

@@ -0,0 +1,172 @@
<div align="center">
<h1>
<img src="./assets/fire.svg" width=30, height=30>
𝚃𝚎𝚡𝚃𝚎𝚕𝚕𝚎𝚛
<img src="./assets/fire.svg" width=30, height=30>
</h1>
<p align="center">
English | <a href="./assets/README_zh.md">中文</a>
</p>
<!-- <p align="center">
<img src="./assets/web_demo.gif" alt="TexTeller_demo" width=800>
</p> -->
</div>
https://github.com/OleehyO/TexTeller/assets/56267907/b23b2b2e-a663-4abb-b013-bd47238d513b
TexTeller is an end-to-end formula recognition model based on ViT, capable of converting images into corresponding LaTeX formulas.
TexTeller was trained with ~~550K~~7.5M image-formula pairs (dataset available [here](https://huggingface.co/datasets/OleehyO/latex-formulas)), compared to [LaTeX-OCR](https://github.com/lukas-blecher/LaTeX-OCR) which used a 100K dataset, TexTeller has **stronger generalization abilities** and **higher accuracy**, covering most use cases (**except for scanned images and handwritten formulas**).
> ~~We will soon release a TexTeller checkpoint trained on a 7.5M dataset~~
## 🔄 Change Log
* 📮[2024-03-24] TexTeller 2.0 released! The training data for TexTeller 2.0 has been increased to 7.5M (about **15 times more** than TexTeller 1.0 and also improved in data quality). The trained TexTeller 2.0 demonstrated **superior performance** in the test set, especially in recognizing rare symbols, complex multi-line formulas, and matrices.
> [There](./assets/test.pdf) are more test images here and a horizontal comparison of recognition models from different companies.
## 🔑 Prerequisites
python=3.10
pytorch
> [!WARNING]
> Only CUDA versions >= 12.0 have been fully tested, so it is recommended to use CUDA version >= 12.0
## 🖼 About Rendering LaTeX as Images
* **Install XeLaTex** and ensure `xelatex` can be called directly from the command line.
* To ensure correct rendering of the predicted formulas, **include the following packages** in your `.tex` file:
```tex
\usepackage{multirow,multicol,amsmath,amsfonts,amssymb,mathtools,bm,mathrsfs,wasysym,amsbsy,upgreek,mathalfa,stmaryrd,mathrsfs,dsfont,amsthm,amsmath,multirow}
```
## 🚀 Getting Started
1. Clone the repository:
```bash
git clone https://github.com/OleehyO/TexTeller
```
2. After [installing pytorch](https://pytorch.org/get-started/locally/#start-locally), install the project's dependencies:
```bash
pip install -r requirements.txt
```
3. Enter the `TexTeller/src` directory and run the following command in the terminal to start inference:
```bash
python inference.py -img "/path/to/image.{jpg,png}"
# use -cuda option to enable GPU inference
#+e.g. python inference.py -img "./img.jpg" -cuda
```
> [!NOTE]
> The first time you run it, the required checkpoints will be downloaded from Hugging Face
## 🌐 Web Demo
To start the web demo, you need to first enter the `TexTeller/src` directory, then run the following command
```bash
./start_web.sh
```
Then, enter `http://localhost:8501` in your browser to see the web demo
> [!TIP]
> You can change the default configuration of `start_web.sh`, for example, to use GPU for inference (e.g. `USE_CUDA=True`) or to increase the number of beams (e.g. `NUM_BEAM=3`) to achieve higher accuracy
> [!IMPORTANT]
> If you want to directly render the prediction results as images on the web (for example, to check if the prediction is correct), you need to ensure [xelatex is correctly installed](https://github.com/OleehyO/TexTeller?tab=readme-ov-file#Rendering-Predicted-Results)
## 📡 API Usage
We use [ray serve](https://github.com/ray-project/ray) to provide an API interface for TexTeller, allowing you to integrate TexTeller into your own projects. To start the server, you first need to enter the `TexTeller/src` directory and then run the following command:
```bash
python server.py # default settings
```
You can pass the following arguments to `server.py` to change the server's inference settings (e.g. `python server.py --use_gpu` to enable GPU inference):
| Parameter | Description |
| --- | --- |
| `-ckpt` | The path to the weights file, *default is TexTeller's pretrained weights*.|
| `-tknz` | The path to the tokenizer, *default is TexTeller's tokenizer*.|
| `-port` | The server's service port, *default is 8000*. |
| `--use_gpu` | Whether to use GPU for inference, *default is CPU*. |
| `--num_beams` | The number of beams for beam search, *default is 1*. |
| `--num_replicas` | The number of service replicas to run on the server, *default is 1 replica*. You can use more replicas to achieve greater throughput.|
| `--ncpu_per_replica` | The number of CPU cores used per service replica, *default is 1*. |
| `--ngpu_per_replica` | The number of GPUs used per service replica, *default is 1*. You can set this value between 0 and 1 to run multiple service replicas on one GPU to share the GPU, thereby improving GPU utilization. (Note, if --num_replicas is 2, --ngpu_per_replica is 0.7, then 2 GPUs must be available) |
> [!NOTE]
> A client demo can be found at `TexTeller/client/demo.py`, you can refer to `demo.py` to send requests to the server
## 🏋️‍♂️ Training
### Dataset
We provide an example dataset in the `TexTeller/src/models/ocr_model/train/dataset` directory, you can place your own images in the `images` directory and annotate each image with its corresponding formula in `formulas.jsonl`.
After preparing your dataset, you need to **change the `DIR_URL` variable to your own dataset's path** in `.../dataset/loader.py`
### Retraining the Tokenizer
If you are using a different dataset, you might need to retrain the tokenizer to obtain a different dictionary. After configuring your dataset, you can train your own tokenizer with the following command:
1. In `TexTeller/src/models/tokenizer/train.py`, change `new_tokenizer.save_pretrained('./your_dir_name')` to your custom output directory
> If you want to use a different dictionary size (default is 10k tokens), you need to change the `VOCAB_SIZE` variable in `TexTeller/src/models/globals.py`
2. **In the `TexTeller/src` directory**, run the following command:
```bash
python -m models.tokenizer.train
```
### Training the Model
To train the model, you need to run the following command in the `TexTeller/src` directory:
```bash
python -m models.ocr_model.train.train
```
You can set your own tokenizer and checkpoint paths in `TexTeller/src/models/ocr_model/train/train.py` (refer to `train.py` for more information). If you are using the same architecture and dictionary as TexTeller, you can also fine-tune TexTeller's default weights with your own dataset.
In `TexTeller/src/globals.py` and `TexTeller/src/models/ocr_model/train/train_args.py`, you can change the model's architecture and training hyperparameters.
> [!NOTE]
> Our training scripts use the [Hugging Face Transformers](https://github.com/huggingface/transformers) library, so you can refer to their [documentation](https://huggingface.co/docs/transformers/v4.32.1/main_classes/trainer#transformers.TrainingArguments) for more details and configurations on training parameters.
## 🚧 Limitations
* Does not support scanned images and PDF document recognition
* Does not support handwritten formulas
## 📅 Plans
- [x] ~~Train the model with a larger dataset (7.5M samples, coming soon)~~
- [ ] Recognition of scanned images
- [ ] PDF document recognition + Support for English and Chinese scenarios
- [ ] Inference acceleration
- [ ] ...
## 💖 Acknowledgments
Thanks to [LaTeX-OCR](https://github.com/lukas-blecher/LaTeX-OCR) which has brought me a lot of inspiration, and [im2latex-100K](https://zenodo.org/records/56198#.V2px0jXT6eA) which enriches our dataset.
## ⭐️ Stargazers over time
[![Stargazers over time](https://starchart.cc/OleehyO/TexTeller.svg?variant=adaptive)](https://starchart.cc/OleehyO/TexTeller)

201
assets/README_zh.md Normal file
View File

@@ -0,0 +1,201 @@
<div align="center">
<h1>
<img src="./fire.svg" width=30, height=30>
𝚃𝚎𝚡𝚃𝚎𝚕𝚕𝚎𝚛
<img src="./fire.svg" width=30, height=30>
</h1>
<p align="center">
<a href="../README.md">English</a> | 中文
</p>
<!-- <p align="center">
<img src="./web_demo.gif" alt="TexTeller_demo" width=800>
</p> -->
</div>
https://github.com/OleehyO/TexTeller/assets/56267907/fb17af43-f2a5-47ce-ad1d-101db5fd7fbb
TexTeller是一个基于ViT的端到端公式识别模型，可以把图片转换为对应的latex公式
TexTeller用了~~550K~~7.5M的图片-公式对进行训练(数据集可以在[这里](https://huggingface.co/datasets/OleehyO/latex-formulas)获取)，相比于[LaTeX-OCR](https://github.com/lukas-blecher/LaTeX-OCR)(使用了一个100K的数据集)，TexTeller具有**更强的泛化能力**以及**更高的准确率**，可以覆盖大部分的使用场景(**扫描图片，手写公式除外**)。
> ~~我们马上就会发布一个使用7.5M数据集进行训练的TexTeller checkpoint~~
## 🔄 变更信息
* 📮[2024-03-24] TexTeller2.0发布！TexTeller2.0的训练数据增大到了7.5M(相较于TexTeller1.0**增加了~15倍**并且数据质量也有所改善)。训练后的TexTeller2.0在测试集中展现出了**更加优越的性能**，尤其在生僻符号、复杂多行、矩阵的识别场景中。
> 在[这里](./test.pdf)有更多的测试图片以及各家识别模型的横向对比。
## 🔑 前置条件
python=3.10
pytorch
> [!WARNING]
> 只有CUDA版本>= 12.0被完全测试过，所以最好使用>= 12.0的CUDA版本
## 🖼 关于把latex渲染成图片
* **安装XeLaTex** 并确保`xelatex`可以直接被命令行调用。
* 为了确保正确渲染预测出的公式, 需要在`.tex`文件中**引入以下宏包**:
```tex
\usepackage{multirow,multicol,amsmath,amsfonts,amssymb,mathtools,bm,mathrsfs,wasysym,amsbsy,upgreek,mathalfa,stmaryrd,mathrsfs,dsfont,amsthm,amsmath,multirow}
```
## 🚀 开搞
1. 克隆本仓库:
```bash
git clone https://github.com/OleehyO/TexTeller
```
2. [安装pytorch](https://pytorch.org/get-started/locally/#start-locally)后，再安装本项目的依赖包:
```bash
pip install -r requirements.txt
```
3. 进入`TexTeller/src`目录，在终端运行以下命令进行推理:
```bash
python inference.py -img "/path/to/image.{jpg,png}"
# use -cuda option to enable GPU inference
#+e.g. python inference.py -img "./img.jpg" -cuda
```
> [!NOTE]
> 第一次运行时会在hugging face上下载所需要的checkpoints
## ❓ 常见问题：无法连接到Hugging Face
默认情况下，会在Hugging Face中下载模型权重，**如果你的远端服务器无法连接到Hugging Face**，你可以通过以下命令进行加载：
1. 安装huggingface hub包
```bash
pip install -U "huggingface_hub[cli]"
```
2. 在能连接Hugging Face的机器上下载模型权重:
```bash
huggingface-cli download OleehyO/TexTeller --include "*.json" "*.bin" "*.txt" --repo-type model --local-dir "your/dir/path"
```
3. 把包含权重的目录上传远端服务器，然后把`TexTeller/src/models/ocr_model/model/TexTeller.py`中的`REPO_NAME = 'OleehyO/TexTeller'`修改为`REPO_NAME = 'your/dir/path'`
如果你还想在训练模型时开启evaluate，你需要提前下载metric脚本并上传远端服务器：
1. 在能连接Hugging Face的机器上下载metric脚本
```bash
huggingface-cli download evaluate-metric/google_bleu --repo-type space --local-dir "your/dir/path"
```
2. 把这个目录上传远端服务器，并在`TexTeller/src/models/ocr_model/utils/metrics.py`中把`evaluate.load('google_bleu')`改为`evaluate.load('your/dir/path/google_bleu.py')`
## 🌐 网页演示
要想启动web demo，你需要先进入 `TexTeller/src` 目录，然后运行以下命令
```bash
./start_web.sh
```
然后在浏览器里输入`http://localhost:8501`就可以看到web demo
> [!TIP]
> 你可以改变`start_web.sh`的默认配置， 例如使用GPU进行推理(e.g. `USE_CUDA=True`) 或者增加beams的数量(e.g. `NUM_BEAM=3`)来获得更高的精确度
> [!IMPORTANT]
> 如果你想直接把预测结果在网页上渲染成图片（比如为了检查预测结果是否正确）你需要确保[xelatex被正确安装](https://github.com/OleehyO/TexTeller?tab=readme-ov-file#Rendering-Predicted-Results)
## 📡 API调用
我们使用[ray serve](https://github.com/ray-project/ray)来对外提供一个TexTeller的API接口，通过使用这个接口，你可以把TexTeller整合到自己的项目里。要想启动server，你需要先进入`TexTeller/src`目录然后运行以下命令:
```bash
python server.py # default settings
```
你可以给`server.py`传递以下参数来改变server的推理设置(e.g. `python server.py --use_gpu` 来启动GPU推理):
| 参数 | 描述 |
| --- | --- |
| `-ckpt` | 权重文件的路径，*默认为TexTeller的预训练权重*。|
| `-tknz` | 分词器的路径， *默认为TexTeller的分词器*。|
| `-port` | 服务器的服务端口， *默认是8000*。 |
| `--use_gpu` | 是否使用GPU推理，*默认为CPU*。 |
| `--num_beams` | beam search的beam数量， *默认是1*。 |
| `--num_replicas` | 在服务器上运行的服务副本数量， *默认1个副本*。你可以使用更多的副本来获取更大的吞吐量。|
| `--ncpu_per_replica` | 每个服务副本所用的CPU核心数，*默认为1*。 |
| `--ngpu_per_replica` | 每个服务副本所用的GPU数量，*默认为1*。你可以把这个值设置成 0~1之间的数，这样会在一个GPU上运行多个服务副本来共享GPU，从而提高GPU的利用率。(注意，如果 --num_replicas 2, --ngpu_per_replica 0.7, 那么就必须要有2个GPU可用) |
> [!NOTE]
> 一个客户端demo可以在`TexTeller/client/demo.py`找到，你可以参考`demo.py`来给server发送请求
## 🏋️‍♂️ 训练
### 数据集
我们在`TexTeller/src/models/ocr_model/train/dataset`目录中提供了一个数据集的例子，你可以把自己的图片放在`images`目录然后在`formulas.jsonl`中为每张图片标注对应的公式。
准备好数据集后，你需要在`.../dataset/loader.py`中把 **`DIR_URL`变量改成你自己数据集的路径**
### 重新训练分词器
如果你使用了不一样的数据集，你可能需要重新训练tokenizer来得到一个不一样的字典。配置好数据集后，可以通过以下命令来训练自己的tokenizer：
1. 在`TexTeller/src/models/tokenizer/train.py`中，修改`new_tokenizer.save_pretrained('./your_dir_name')`为你自定义的输出目录
> [!IMPORTANT]
> 如果要用一个不一样大小的字典(默认1W个token)，你需要在 `TexTeller/src/models/globals.py`中修改`VOCAB_SIZE`变量
2. **在 `TexTeller/src` 目录下**运行以下命令:
```bash
python -m models.tokenizer.train
```
### 训练模型
要想训练模型, 你需要在`TexTeller/src`目录下运行以下命令：
```bash
python -m models.ocr_model.train.train
```
你可以在`TexTeller/src/models/ocr_model/train/train.py`中设置自己的tokenizer和checkpoint路径（请参考`train.py`）。如果你使用了与TexTeller一样的架构和相同的字典，你还可以用自己的数据集来微调TexTeller的默认权重。
在`TexTeller/src/globals.py`和`TexTeller/src/models/ocr_model/train/train_args.py`中，你可以改变模型的架构以及训练的超参数。
> [!NOTE]
> 我们的训练脚本使用了[Hugging Face Transformers](https://github.com/huggingface/transformers)库, 所以你可以参考他们提供的[文档](https://huggingface.co/docs/transformers/v4.32.1/main_classes/trainer#transformers.TrainingArguments)来获取更多训练参数的细节以及配置。
## 🚧 不足
* 不支持扫描图片以及PDF文档识别
* 不支持手写体公式
## 📅 计划
- [x] ~~使用更大的数据集来训练模型(7.5M样本，即将发布)~~
- [ ] 扫描图片识别
- [ ] PDF文档识别 + 中英文场景支持
- [ ] 推理加速
- [ ] ...
## 💖 感谢
Thanks to [LaTeX-OCR](https://github.com/lukas-blecher/LaTeX-OCR) which has brought me a lot of inspiration, and [im2latex-100K](https://zenodo.org/records/56198#.V2px0jXT6eA) which enriches our dataset.
## ⭐️ 观星曲线
[![Stargazers over time](https://starchart.cc/OleehyO/TexTeller.svg?variant=adaptive)](https://starchart.cc/OleehyO/TexTeller)

460
assets/fire.svg Normal file
View File

@@ -0,0 +1,460 @@
<svg xmlns="http://www.w3.org/2000/svg" xmlns:xlink="http://www.w3.org/1999/xlink" style="" width="200px" height="100px" viewBox="0 0 100 100" preserveAspectRatio="xMidYMid">
<defs>
<filter id="ldio-ekpf7uvh2aq-filter" filterUnits="userSpaceOnUse" x="0" y="0" width="100" height="100">
<feGaussianBlur in="SourceGraphic" stdDeviation="3"></feGaussianBlur>
<feComponentTransfer result="cutoff">
<feFuncA type="linear" slope="10" intercept="-5"></feFuncA>
</feComponentTransfer>
</filter>
</defs><g filter="url(#ldio-ekpf7uvh2aq-filter)"><circle cx="45" cy="154.67770829199992" r="42" fill="#e15b64">
<animate attributeName="cy" values="154.67770829199992;-27.568110790210763" keyTimes="0;1" dur="1s" repeatCount="indefinite" begin="-0.7914508173328552s"></animate>
<animate attributeName="r" values="42;0;0" keyTimes="0;0.6593879177915443;1" dur="1s" repeatCount="indefinite" begin="-0.7914508173328552s"></animate>
</circle><circle cx="53" cy="156.51873756667007" r="43" fill="#e15b64">
<animate attributeName="cy" values="156.51873756667007;-28.593472199379597" keyTimes="0;1" dur="1s" repeatCount="indefinite" begin="-0.8990601299952956s"></animate>
<animate attributeName="r" values="43;0;0" keyTimes="0;0.9199190750649376;1" dur="1s" repeatCount="indefinite" begin="-0.8990601299952956s"></animate>
</circle><circle cx="22" cy="118.4676277511406" r="6" fill="#e15b64">
<animate attributeName="cy" values="118.4676277511406;-1.812134766063739" keyTimes="0;1" dur="1s" repeatCount="indefinite" begin="-0.2574158626531723s"></animate>
<animate attributeName="r" values="6;0;0" keyTimes="0;0.7424894336620584;1" dur="1s" repeatCount="indefinite" begin="-0.2574158626531723s"></animate>
</circle><circle cx="56" cy="143.3980016480395" r="34" fill="#e15b64">
<animate attributeName="cy" values="143.3980016480395;-23.264651741765398" keyTimes="0;1" dur="1s" repeatCount="indefinite" begin="-0.5292591072219247s"></animate>
<animate attributeName="r" values="34;0;0" keyTimes="0;0.8257208789488842;1" dur="1s" repeatCount="indefinite" begin="-0.5292591072219247s"></animate>
</circle><circle cx="43" cy="154.61226210156264" r="43" fill="#e15b64">
<animate attributeName="cy" values="154.61226210156264;-39.72257238426019" keyTimes="0;1" dur="1s" repeatCount="indefinite" begin="-0.9349241678635103s"></animate>
<animate attributeName="r" values="43;0;0" keyTimes="0;0.6655411648349204;1" dur="1s" repeatCount="indefinite" begin="-0.9349241678635103s"></animate>
</circle><circle cx="36" cy="141.18233539125538" r="23" fill="#e15b64">
<animate attributeName="cy" values="141.18233539125538;-11.919782601799477" keyTimes="0;1" dur="1s" repeatCount="indefinite" begin="-0.9661184430026497s"></animate>
<animate attributeName="r" values="23;0;0" keyTimes="0;0.7340510315067473;1" dur="1s" repeatCount="indefinite" begin="-0.9661184430026497s"></animate>
</circle><circle cx="55" cy="137.61381349909033" r="35" fill="#e15b64">
<animate attributeName="cy" values="137.61381349909033;-27.023105799592948" keyTimes="0;1" dur="1s" repeatCount="indefinite" begin="-0.7882390392923937s"></animate>
<animate attributeName="r" values="35;0;0" keyTimes="0;0.5596286394923506;1" dur="1s" repeatCount="indefinite" begin="-0.7882390392923937s"></animate>
</circle><circle cx="81" cy="116.42482869722863" r="6" fill="#e15b64">
<animate attributeName="cy" values="116.42482869722863;2.642571962973477" keyTimes="0;1" dur="1s" repeatCount="indefinite" begin="-0.6838551001109257s"></animate>
<animate attributeName="r" values="6;0;0" keyTimes="0;0.8530428185299654;1" dur="1s" repeatCount="indefinite" begin="-0.6838551001109257s"></animate>
</circle><circle cx="51" cy="144.1337397120671" r="41" fill="#e15b64">
<animate attributeName="cy" values="144.1337397120671;-35.62888188299487" keyTimes="0;1" dur="1s" repeatCount="indefinite" begin="-0.8931867510460544s"></animate>
<animate attributeName="r" values="41;0;0" keyTimes="0;0.9351064787950636;1" dur="1s" repeatCount="indefinite" begin="-0.8931867510460544s"></animate>
</circle><circle cx="22" cy="127.94124738258117" r="20" fill="#e15b64">
<animate attributeName="cy" values="127.94124738258117;-4.588101238414598" keyTimes="0;1" dur="1s" repeatCount="indefinite" begin="-0.9129507531699166s"></animate>
<animate attributeName="r" values="20;0;0" keyTimes="0;0.9626971761152365;1" dur="1s" repeatCount="indefinite" begin="-0.9129507531699166s"></animate>
</circle><circle cx="51" cy="130.13871763314205" r="21" fill="#e15b64">
<animate attributeName="cy" values="130.13871763314205;-2.771870373434613" keyTimes="0;1" dur="1s" repeatCount="indefinite" begin="-0.16276671760313832s"></animate>
<animate attributeName="r" values="21;0;0" keyTimes="0;0.6367210977937845;1" dur="1s" repeatCount="indefinite" begin="-0.16276671760313832s"></animate>
</circle><circle cx="28" cy="130.94671647108635" r="26" fill="#e15b64">
<animate attributeName="cy" values="130.94671647108635;-20.54470862263146" keyTimes="0;1" dur="1s" repeatCount="indefinite" begin="-0.010777607623041363s"></animate>
<animate attributeName="r" values="26;0;0" keyTimes="0;0.5986827903483527;1" dur="1s" repeatCount="indefinite" begin="-0.010777607623041363s"></animate>
</circle><circle cx="32" cy="133.57559887485095" r="18" fill="#e15b64">
<animate attributeName="cy" values="133.57559887485095;-13.998747273650661" keyTimes="0;1" dur="1s" repeatCount="indefinite" begin="-0.6849903294560423s"></animate>
<animate attributeName="r" values="18;0;0" keyTimes="0;0.9272684317035897;1" dur="1s" repeatCount="indefinite" begin="-0.6849903294560423s"></animate>
</circle><circle cx="50" cy="129.2368025879272" r="29" fill="#e15b64">
<animate attributeName="cy" values="129.2368025879272;-21.38222818211007" keyTimes="0;1" dur="1s" repeatCount="indefinite" begin="-0.2570532837614655s"></animate>
<animate attributeName="r" values="29;0;0" keyTimes="0;0.5349692982819836;1" dur="1s" repeatCount="indefinite" begin="-0.2570532837614655s"></animate>
</circle><circle cx="54" cy="147.67203918209864" r="32" fill="#e15b64">
<animate attributeName="cy" values="147.67203918209864;-23.292000640460095" keyTimes="0;1" dur="1s" repeatCount="indefinite" begin="-0.8840781999829185s"></animate>
<animate attributeName="r" values="32;0;0" keyTimes="0;0.9905440228534627;1" dur="1s" repeatCount="indefinite" begin="-0.8840781999829185s"></animate>
</circle><circle cx="49" cy="156.33097983975816" r="43" fill="#e15b64">
<animate attributeName="cy" values="156.33097983975816;-30.688836209655307" keyTimes="0;1" dur="1s" repeatCount="indefinite" begin="-0.6363282840605137s"></animate>
<animate attributeName="r" values="43;0;0" keyTimes="0;0.578321371334853;1" dur="1s" repeatCount="indefinite" begin="-0.6363282840605137s"></animate>
</circle><circle cx="53" cy="150.73132612778645" r="38" fill="#e15b64">
<animate attributeName="cy" values="150.73132612778645;-24.243875812169208" keyTimes="0;1" dur="1s" repeatCount="indefinite" begin="-0.6889884148164682s"></animate>
<animate attributeName="r" values="38;0;0" keyTimes="0;0.9820908894527897;1" dur="1s" repeatCount="indefinite" begin="-0.6889884148164682s"></animate>
</circle><circle cx="58" cy="136.92364235316566" r="30" fill="#e15b64">
<animate attributeName="cy" values="136.92364235316566;-14.514104757207221" keyTimes="0;1" dur="1s" repeatCount="indefinite" begin="-0.3274028295945308s"></animate>
<animate attributeName="r" values="30;0;0" keyTimes="0;0.9109990458833535;1" dur="1s" repeatCount="indefinite" begin="-0.3274028295945308s"></animate>
</circle><circle cx="21" cy="125.47085228007643" r="18" fill="#e15b64">
<animate attributeName="cy" values="125.47085228007643;-8.232426956653288" keyTimes="0;1" dur="1s" repeatCount="indefinite" begin="-0.11103461733078768s"></animate>
<animate attributeName="r" values="18;0;0" keyTimes="0;0.7718042613876622;1" dur="1s" repeatCount="indefinite" begin="-0.11103461733078768s"></animate>
</circle><circle cx="57" cy="154.13251799723747" r="37" fill="#e15b64">
<animate attributeName="cy" values="154.13251799723747;-18.665203993986026" keyTimes="0;1" dur="1s" repeatCount="indefinite" begin="-0.8263441768461145s"></animate>
<animate attributeName="r" values="37;0;0" keyTimes="0;0.7148325280461965;1" dur="1s" repeatCount="indefinite" begin="-0.8263441768461145s"></animate>
</circle><circle cx="52" cy="163.55969451733722" r="47" fill="#e15b64">
<animate attributeName="cy" values="163.55969451733722;-45.32343944696123" keyTimes="0;1" dur="1s" repeatCount="indefinite" begin="-0.08605155305311041s"></animate>
<animate attributeName="r" values="47;0;0" keyTimes="0;0.8554524873372089;1" dur="1s" repeatCount="indefinite" begin="-0.08605155305311041s"></animate>
</circle><circle cx="43" cy="150.72861891310126" r="42" fill="#e15b64">
<animate attributeName="cy" values="150.72861891310126;-23.942286768617272" keyTimes="0;1" dur="1s" repeatCount="indefinite" begin="-0.8013052401764136s"></animate>
<animate attributeName="r" values="42;0;0" keyTimes="0;0.6681090498432822;1" dur="1s" repeatCount="indefinite" begin="-0.8013052401764136s"></animate>
</circle><circle cx="62" cy="109.2607457626771" r="2" fill="#e15b64">
<animate attributeName="cy" values="109.2607457626771;3.194634855160243" keyTimes="0;1" dur="1s" repeatCount="indefinite" begin="-0.7901767326521292s"></animate>
<animate attributeName="r" values="2;0;0" keyTimes="0;0.7018579919397697;1" dur="1s" repeatCount="indefinite" begin="-0.7901767326521292s"></animate>
</circle><circle cx="29" cy="132.04950518708117" r="26" fill="#e15b64">
<animate attributeName="cy" values="132.04950518708117;-24.268419710129816" keyTimes="0;1" dur="1s" repeatCount="indefinite" begin="-0.9729317633977274s"></animate>
<animate attributeName="r" values="26;0;0" keyTimes="0;0.8277305604086497;1" dur="1s" repeatCount="indefinite" begin="-0.9729317633977274s"></animate>
</circle><circle cx="54" cy="150.69697127653222" r="41" fill="#e15b64">
<animate attributeName="cy" values="150.69697127653222;-27.168516505190766" keyTimes="0;1" dur="1s" repeatCount="indefinite" begin="-0.5902016146688314s"></animate>
<animate attributeName="r" values="41;0;0" keyTimes="0;0.8175867220161461;1" dur="1s" repeatCount="indefinite" begin="-0.5902016146688314s"></animate>
</circle><circle cx="50" cy="115.01352405454155" r="7" fill="#e15b64">
<animate attributeName="cy" values="115.01352405454155;-4.5076288690789195" keyTimes="0;1" dur="1s" repeatCount="indefinite" begin="-0.5091907734741129s"></animate>
<animate attributeName="r" values="7;0;0" keyTimes="0;0.6751846924914742;1" dur="1s" repeatCount="indefinite" begin="-0.5091907734741129s"></animate>
</circle><circle cx="65" cy="137.6419430633514" r="34" fill="#e15b64">
<animate attributeName="cy" values="137.6419430633514;-17.00344965868893" keyTimes="0;1" dur="1s" repeatCount="indefinite" begin="-0.34747192063247945s"></animate>
<animate attributeName="r" values="34;0;0" keyTimes="0;0.5212737600536792;1" dur="1s" repeatCount="indefinite" begin="-0.34747192063247945s"></animate>
</circle><circle cx="34" cy="127.0455079544209" r="14" fill="#e15b64">
<animate attributeName="cy" values="127.0455079544209;-3.6990759299641454" keyTimes="0;1" dur="1s" repeatCount="indefinite" begin="-0.4890615261218786s"></animate>
<animate attributeName="r" values="14;0;0" keyTimes="0;0.6183470012170013;1" dur="1s" repeatCount="indefinite" begin="-0.4890615261218786s"></animate>
</circle><circle cx="12" cy="120.43345098845494" r="3" fill="#e15b64">
<animate attributeName="cy" values="120.43345098845494;9.74374931913883" keyTimes="0;1" dur="1s" repeatCount="indefinite" begin="-0.3026505339978601s"></animate>
<animate attributeName="r" values="3;0;0" keyTimes="0;0.5414300978949788;1" dur="1s" repeatCount="indefinite" begin="-0.3026505339978601s"></animate>
</circle><circle cx="49" cy="161.35205628493102" r="43" fill="#e15b64">
<animate attributeName="cy" values="161.35205628493102;-37.872089939512506" keyTimes="0;1" dur="1s" repeatCount="indefinite" begin="-0.38741962448531564s"></animate>
<animate attributeName="r" values="43;0;0" keyTimes="0;0.5096615889177538;1" dur="1s" repeatCount="indefinite" begin="-0.38741962448531564s"></animate>
</circle><circle cx="54" cy="146.5769009919314" r="44" fill="#e15b64">
<animate attributeName="cy" values="146.5769009919314;-38.33530354334875" keyTimes="0;1" dur="1s" repeatCount="indefinite" begin="-0.34335748774106034s"></animate>
<animate attributeName="r" values="44;0;0" keyTimes="0;0.743420827137904;1" dur="1s" repeatCount="indefinite" begin="-0.34335748774106034s"></animate>
</circle><circle cx="20" cy="111.24659457696168" r="7" fill="#e15b64">
<animate attributeName="cy" values="111.24659457696168;10.851798254886354" keyTimes="0;1" dur="1s" repeatCount="indefinite" begin="-0.6282307990647713s"></animate>
<animate attributeName="r" values="7;0;0" keyTimes="0;0.8297799829349941;1" dur="1s" repeatCount="indefinite" begin="-0.6282307990647713s"></animate>
</circle><circle cx="50" cy="164.0676485495781" r="45" fill="#e15b64">
<animate attributeName="cy" values="164.0676485495781;-31.499414285176986" keyTimes="0;1" dur="1s" repeatCount="indefinite" begin="-0.7760446285439819s"></animate>
<animate attributeName="r" values="45;0;0" keyTimes="0;0.5740694195049653;1" dur="1s" repeatCount="indefinite" begin="-0.7760446285439819s"></animate>
</circle><circle cx="63" cy="121.15583070803987" r="16" fill="#e15b64">
<animate attributeName="cy" values="121.15583070803987;-2.1042758907266066" keyTimes="0;1" dur="1s" repeatCount="indefinite" begin="-0.2305276534763374s"></animate>
<animate attributeName="r" values="16;0;0" keyTimes="0;0.5205278426126575;1" dur="1s" repeatCount="indefinite" begin="-0.2305276534763374s"></animate>
</circle><circle cx="70" cy="143.94247592516618" r="29" fill="#e15b64">
<animate attributeName="cy" values="143.94247592516618;-23.62297573618442" keyTimes="0;1" dur="1s" repeatCount="indefinite" begin="-0.5284797120514513s"></animate>
<animate attributeName="r" values="29;0;0" keyTimes="0;0.9336811516026573;1" dur="1s" repeatCount="indefinite" begin="-0.5284797120514513s"></animate>
</circle><circle cx="21" cy="122.79868387744153" r="20" fill="#e15b64">
<animate attributeName="cy" values="122.79868387744153;-13.104461771681535" keyTimes="0;1" dur="1s" repeatCount="indefinite" begin="-0.8845782118773111s"></animate>
<animate attributeName="r" values="20;0;0" keyTimes="0;0.904216846935756;1" dur="1s" repeatCount="indefinite" begin="-0.8845782118773111s"></animate>
</circle><circle cx="46" cy="143.70707265719267" r="24" fill="#e15b64">
<animate attributeName="cy" values="143.70707265719267;-20.28891701845349" keyTimes="0;1" dur="1s" repeatCount="indefinite" begin="-0.23245576862802375s"></animate>
<animate attributeName="r" values="24;0;0" keyTimes="0;0.6586288079548765;1" dur="1s" repeatCount="indefinite" begin="-0.23245576862802375s"></animate>
</circle><circle cx="65" cy="140.13731645312657" r="22" fill="#e15b64">
<animate attributeName="cy" values="140.13731645312657;-5.338876455584764" keyTimes="0;1" dur="1s" repeatCount="indefinite" begin="-0.7182419259629308s"></animate>
<animate attributeName="r" values="22;0;0" keyTimes="0;0.8813907372203135;1" dur="1s" repeatCount="indefinite" begin="-0.7182419259629308s"></animate>
</circle><circle cx="37" cy="139.00958710472267" r="35" fill="#e15b64">
<animate attributeName="cy" values="139.00958710472267;-25.68265144780311" keyTimes="0;1" dur="1s" repeatCount="indefinite" begin="-0.7030100698848409s"></animate>
<animate attributeName="r" values="35;0;0" keyTimes="0;0.7320613459176248;1" dur="1s" repeatCount="indefinite" begin="-0.7030100698848409s"></animate>
</circle><circle cx="45" cy="146.6744507961619" r="44" fill="#e15b64">
<animate attributeName="cy" values="146.6744507961619;-38.087338695486295" keyTimes="0;1" dur="1s" repeatCount="indefinite" begin="-0.8319540053556033s"></animate>
<animate attributeName="r" values="44;0;0" keyTimes="0;0.5904241586083279;1" dur="1s" repeatCount="indefinite" begin="-0.8319540053556033s"></animate>
</circle><circle cx="53" cy="116.16529146873187" r="15" fill="#e15b64">
<animate attributeName="cy" values="116.16529146873187;-3.17669223153381" keyTimes="0;1" dur="1s" repeatCount="indefinite" begin="-0.7864341362651808s"></animate>
<animate attributeName="r" values="15;0;0" keyTimes="0;0.589186107816807;1" dur="1s" repeatCount="indefinite" begin="-0.7864341362651808s"></animate>
</circle><circle cx="29" cy="141.6902909599232" r="23" fill="#e15b64">
<animate attributeName="cy" values="141.6902909599232;-16.250272669063218" keyTimes="0;1" dur="1s" repeatCount="indefinite" begin="-0.18084365714200346s"></animate>
<animate attributeName="r" values="23;0;0" keyTimes="0;0.8116571311237253;1" dur="1s" repeatCount="indefinite" begin="-0.18084365714200346s"></animate>
</circle><circle cx="65" cy="143.73302386926983" r="32" fill="#e15b64">
<animate attributeName="cy" values="143.73302386926983;-24.229369251904558" keyTimes="0;1" dur="1s" repeatCount="indefinite" begin="-0.5786484558188305s"></animate>
<animate attributeName="r" values="32;0;0" keyTimes="0;0.8515606125902615;1" dur="1s" repeatCount="indefinite" begin="-0.5786484558188305s"></animate>
</circle><circle cx="39" cy="143.3951504366216" r="33" fill="#e15b64">
<animate attributeName="cy" values="143.3951504366216;-27.75171362166084" keyTimes="0;1" dur="1s" repeatCount="indefinite" begin="-0.1481578769905092s"></animate>
<animate attributeName="r" values="33;0;0" keyTimes="0;0.797255218191478;1" dur="1s" repeatCount="indefinite" begin="-0.1481578769905092s"></animate>
</circle><circle cx="59" cy="129.28605384114482" r="27" fill="#e15b64">
<animate attributeName="cy" values="129.28605384114482;-12.095864862844131" keyTimes="0;1" dur="1s" repeatCount="indefinite" begin="-0.23581997562886903s"></animate>
<animate attributeName="r" values="27;0;0" keyTimes="0;0.8271538616610963;1" dur="1s" repeatCount="indefinite" begin="-0.23581997562886903s"></animate>
</circle><circle cx="70" cy="144.09835508207823" r="28" fill="#e15b64">
<animate attributeName="cy" values="144.09835508207823;-13.162793363728145" keyTimes="0;1" dur="1s" repeatCount="indefinite" begin="-0.23606519556482253s"></animate>
<animate attributeName="r" values="28;0;0" keyTimes="0;0.73085815703799;1" dur="1s" repeatCount="indefinite" begin="-0.23606519556482253s"></animate>
</circle><circle cx="48" cy="145.01565757702042" r="44" fill="#e15b64">
<animate attributeName="cy" values="145.01565757702042;-32.30510020024561" keyTimes="0;1" dur="1s" repeatCount="indefinite" begin="-0.8615348704203486s"></animate>
<animate attributeName="r" values="44;0;0" keyTimes="0;0.9694373671371078;1" dur="1s" repeatCount="indefinite" begin="-0.8615348704203486s"></animate>
</circle><circle cx="95" cy="113.78554320990165" r="4" fill="#e15b64">
<animate attributeName="cy" values="113.78554320990165;-1.2652564238335904" keyTimes="0;1" dur="1s" repeatCount="indefinite" begin="-0.21370544900580335s"></animate>
<animate attributeName="r" values="4;0;0" keyTimes="0;0.5334621383741172;1" dur="1s" repeatCount="indefinite" begin="-0.21370544900580335s"></animate>
</circle><circle cx="57" cy="136.06708935936715" r="34" fill="#e15b64">
<animate attributeName="cy" values="136.06708935936715;-19.758990054858902" keyTimes="0;1" dur="1s" repeatCount="indefinite" begin="-0.7755376997281404s"></animate>
<animate attributeName="r" values="34;0;0" keyTimes="0;0.9943252777203475;1" dur="1s" repeatCount="indefinite" begin="-0.7755376997281404s"></animate>
</circle><circle cx="72" cy="123.8422572942333" r="19" fill="#e15b64">
<animate attributeName="cy" values="123.8422572942333;-1.0000700639794928" keyTimes="0;1" dur="1s" repeatCount="indefinite" begin="-0.9670461872772004s"></animate>
<animate attributeName="r" values="19;0;0" keyTimes="0;0.7801926792335607;1" dur="1s" repeatCount="indefinite" begin="-0.9670461872772004s"></animate>
</circle></g><g filter="url(#ldio-ekpf7uvh2aq-filter)"><circle cx="27" cy="136.75172282051147" r="17" fill="#f47e60">
<animate attributeName="cy" values="136.75172282051147;-5.48853662281188" keyTimes="0;1" dur="1s" repeatCount="indefinite" begin="-0.4403846891955857s"></animate>
<animate attributeName="r" values="17;0;0" keyTimes="0;0.7894732341719188;1" dur="1s" repeatCount="indefinite" begin="-0.4403846891955857s"></animate>
</circle><circle cx="34" cy="132.08290473906044" r="28" fill="#f47e60">
<animate attributeName="cy" values="132.08290473906044;-16.339029232048958" keyTimes="0;1" dur="1s" repeatCount="indefinite" begin="-0.7882134883361418s"></animate>
<animate attributeName="r" values="28;0;0" keyTimes="0;0.5035175026787356;1" dur="1s" repeatCount="indefinite" begin="-0.7882134883361418s"></animate>
</circle><circle cx="66" cy="127.45606892584162" r="23" fill="#f47e60">
<animate attributeName="cy" values="127.45606892584162;-11.56763185745981" keyTimes="0;1" dur="1s" repeatCount="indefinite" begin="-0.23537267190332678s"></animate>
<animate attributeName="r" values="23;0;0" keyTimes="0;0.7818578332234903;1" dur="1s" repeatCount="indefinite" begin="-0.23537267190332678s"></animate>
</circle><circle cx="29" cy="124.28337961013858" r="15" fill="#f47e60">
<animate attributeName="cy" values="124.28337961013858;0.8461921465181206" keyTimes="0;1" dur="1s" repeatCount="indefinite" begin="-0.30918442080681285s"></animate>
<animate attributeName="r" values="15;0;0" keyTimes="0;0.9741475377259025;1" dur="1s" repeatCount="indefinite" begin="-0.30918442080681285s"></animate>
</circle><circle cx="61" cy="147.91603256008383" r="31" fill="#f47e60">
<animate attributeName="cy" values="147.91603256008383;-14.754981670358578" keyTimes="0;1" dur="1s" repeatCount="indefinite" begin="-0.0033816756583812113s"></animate>
<animate attributeName="r" values="31;0;0" keyTimes="0;0.6463193577485268;1" dur="1s" repeatCount="indefinite" begin="-0.0033816756583812113s"></animate>
</circle><circle cx="25" cy="120.64483537229628" r="9" fill="#f47e60">
<animate attributeName="cy" values="120.64483537229628;-7.193123212298179" keyTimes="0;1" dur="1s" repeatCount="indefinite" begin="-0.6891092543031828s"></animate>
<animate attributeName="r" values="9;0;0" keyTimes="0;0.8637808572418493;1" dur="1s" repeatCount="indefinite" begin="-0.6891092543031828s"></animate>
</circle><circle cx="12" cy="121.18727231753691" r="4" fill="#f47e60">
<animate attributeName="cy" values="121.18727231753691;15.883181236637633" keyTimes="0;1" dur="1s" repeatCount="indefinite" begin="-0.24454851002004097s"></animate>
<animate attributeName="r" values="4;0;0" keyTimes="0;0.8215012014926046;1" dur="1s" repeatCount="indefinite" begin="-0.24454851002004097s"></animate>
</circle><circle cx="58" cy="136.64954415018815" r="19" fill="#f47e60">
<animate attributeName="cy" values="136.64954415018815;-13.637628862199563" keyTimes="0;1" dur="1s" repeatCount="indefinite" begin="-0.7672442553828805s"></animate>
<animate attributeName="r" values="19;0;0" keyTimes="0;0.7534841891330046;1" dur="1s" repeatCount="indefinite" begin="-0.7672442553828805s"></animate>
</circle><circle cx="69" cy="120.72538023727738" r="10" fill="#f47e60">
<animate attributeName="cy" values="120.72538023727738;-5.651458016294906" keyTimes="0;1" dur="1s" repeatCount="indefinite" begin="-0.6587915764098667s"></animate>
<animate attributeName="r" values="10;0;0" keyTimes="0;0.5977129956186352;1" dur="1s" repeatCount="indefinite" begin="-0.6587915764098667s"></animate>
</circle><circle cx="46" cy="122.63158963579554" r="20" fill="#f47e60">
<animate attributeName="cy" values="122.63158963579554;-8.99196405151625" keyTimes="0;1" dur="1s" repeatCount="indefinite" begin="-0.3698350873089088s"></animate>
<animate attributeName="r" values="20;0;0" keyTimes="0;0.5563937567659611;1" dur="1s" repeatCount="indefinite" begin="-0.3698350873089088s"></animate>
</circle><circle cx="7" cy="121.15700947168602" r="2" fill="#f47e60">
<animate attributeName="cy" values="121.15700947168602;0.605011189845321" keyTimes="0;1" dur="1s" repeatCount="indefinite" begin="-0.514133243834255s"></animate>
<animate attributeName="r" values="2;0;0" keyTimes="0;0.7510335363256938;1" dur="1s" repeatCount="indefinite" begin="-0.514133243834255s"></animate>
</circle><circle cx="19" cy="117.69071117783832" r="7" fill="#f47e60">
<animate attributeName="cy" values="117.69071117783832;-2.4512162536532234" keyTimes="0;1" dur="1s" repeatCount="indefinite" begin="-0.4163222368875168s"></animate>
<animate attributeName="r" values="7;0;0" keyTimes="0;0.9697983093212361;1" dur="1s" repeatCount="indefinite" begin="-0.4163222368875168s"></animate>
</circle><circle cx="34" cy="122.22172344680293" r="22" fill="#f47e60">
<animate attributeName="cy" values="122.22172344680293;-14.875000336072436" keyTimes="0;1" dur="1s" repeatCount="indefinite" begin="-0.8346904488502503s"></animate>
<animate attributeName="r" values="22;0;0" keyTimes="0;0.9284864899458874;1" dur="1s" repeatCount="indefinite" begin="-0.8346904488502503s"></animate>
</circle><circle cx="48" cy="118.34245443793573" r="12" fill="#f47e60">
<animate attributeName="cy" values="118.34245443793573;6.1569446890589035" keyTimes="0;1" dur="1s" repeatCount="indefinite" begin="-0.7372012265846987s"></animate>
<animate attributeName="r" values="12;0;0" keyTimes="0;0.9146509122657862;1" dur="1s" repeatCount="indefinite" begin="-0.7372012265846987s"></animate>
</circle><circle cx="38" cy="108.37260349538107" r="4" fill="#f47e60">
<animate attributeName="cy" values="108.37260349538107;-3.9166184571860483" keyTimes="0;1" dur="1s" repeatCount="indefinite" begin="-0.6955752887050161s"></animate>
<animate attributeName="r" values="4;0;0" keyTimes="0;0.9793871272170744;1" dur="1s" repeatCount="indefinite" begin="-0.6955752887050161s"></animate>
</circle><circle cx="50" cy="120.05611377372627" r="20" fill="#f47e60">
<animate attributeName="cy" values="120.05611377372627;-19.59128463520709" keyTimes="0;1" dur="1s" repeatCount="indefinite" begin="-0.8198691615147322s"></animate>
<animate attributeName="r" values="20;0;0" keyTimes="0;0.6017320767396992;1" dur="1s" repeatCount="indefinite" begin="-0.8198691615147322s"></animate>
</circle><circle cx="69" cy="133.11553485199934" r="21" fill="#f47e60">
<animate attributeName="cy" values="133.11553485199934;-7.230262198733577" keyTimes="0;1" dur="1s" repeatCount="indefinite" begin="-0.6502042470386947s"></animate>
<animate attributeName="r" values="21;0;0" keyTimes="0;0.9802383350633911;1" dur="1s" repeatCount="indefinite" begin="-0.6502042470386947s"></animate>
</circle><circle cx="60" cy="138.10205797824347" r="31" fill="#f47e60">
<animate attributeName="cy" values="138.10205797824347;-21.149182634283513" keyTimes="0;1" dur="1s" repeatCount="indefinite" begin="-0.8527464543018912s"></animate>
<animate attributeName="r" values="31;0;0" keyTimes="0;0.5593223005306734;1" dur="1s" repeatCount="indefinite" begin="-0.8527464543018912s"></animate>
</circle><circle cx="72" cy="121.45841247692351" r="16" fill="#f47e60">
<animate attributeName="cy" values="121.45841247692351;-5.0851516529984195" keyTimes="0;1" dur="1s" repeatCount="indefinite" begin="-0.4077549975882817s"></animate>
<animate attributeName="r" values="16;0;0" keyTimes="0;0.5763111141098053;1" dur="1s" repeatCount="indefinite" begin="-0.4077549975882817s"></animate>
</circle><circle cx="56" cy="118.12349945951125" r="10" fill="#f47e60">
<animate attributeName="cy" values="118.12349945951125;-7.082779421666896" keyTimes="0;1" dur="1s" repeatCount="indefinite" begin="-0.21747152423150562s"></animate>
<animate attributeName="r" values="10;0;0" keyTimes="0;0.6868094744383062;1" dur="1s" repeatCount="indefinite" begin="-0.21747152423150562s"></animate>
</circle><circle cx="77" cy="119.41951761904794" r="17" fill="#f47e60">
<animate attributeName="cy" values="119.41951761904794;-9.114276721599797" keyTimes="0;1" dur="1s" repeatCount="indefinite" begin="-0.48345793287516814s"></animate>
<animate attributeName="r" values="17;0;0" keyTimes="0;0.5135663211192452;1" dur="1s" repeatCount="indefinite" begin="-0.48345793287516814s"></animate>
</circle><circle cx="78" cy="125.60192795392818" r="11" fill="#f47e60">
<animate attributeName="cy" values="125.60192795392818;-6.73068982191926" keyTimes="0;1" dur="1s" repeatCount="indefinite" begin="-0.23667812050200931s"></animate>
<animate attributeName="r" values="11;0;0" keyTimes="0;0.9898092475181265;1" dur="1s" repeatCount="indefinite" begin="-0.23667812050200931s"></animate>
</circle><circle cx="51" cy="138.224179154187" r="24" fill="#f47e60">
<animate attributeName="cy" values="138.224179154187;-8.55653503677315" keyTimes="0;1" dur="1s" repeatCount="indefinite" begin="-0.5735700676741093s"></animate>
<animate attributeName="r" values="24;0;0" keyTimes="0;0.9566960986989479;1" dur="1s" repeatCount="indefinite" begin="-0.5735700676741093s"></animate>
</circle><circle cx="41" cy="131.14944604607328" r="21" fill="#f47e60">
<animate attributeName="cy" values="131.14944604607328;-17.847508222350655" keyTimes="0;1" dur="1s" repeatCount="indefinite" begin="-0.07696580759865079s"></animate>
<animate attributeName="r" values="21;0;0" keyTimes="0;0.6865631531399743;1" dur="1s" repeatCount="indefinite" begin="-0.07696580759865079s"></animate>
</circle><circle cx="49" cy="128.787268826053" r="17" fill="#f47e60">
<animate attributeName="cy" values="128.787268826053;1.143259231969072" keyTimes="0;1" dur="1s" repeatCount="indefinite" begin="-0.7890428937034474s"></animate>
<animate attributeName="r" values="17;0;0" keyTimes="0;0.5926722445396657;1" dur="1s" repeatCount="indefinite" begin="-0.7890428937034474s"></animate>
</circle><circle cx="17" cy="120.22416295842616" r="13" fill="#f47e60">
<animate attributeName="cy" values="120.22416295842616;5.932998615440596" keyTimes="0;1" dur="1s" repeatCount="indefinite" begin="-0.25642472915187764s"></animate>
<animate attributeName="r" values="13;0;0" keyTimes="0;0.5738477034101163;1" dur="1s" repeatCount="indefinite" begin="-0.25642472915187764s"></animate>
</circle><circle cx="73" cy="127.02191586426626" r="24" fill="#f47e60">
<animate attributeName="cy" values="127.02191586426626;-19.34982189589097" keyTimes="0;1" dur="1s" repeatCount="indefinite" begin="-0.9257599774553938s"></animate>
<animate attributeName="r" values="24;0;0" keyTimes="0;0.6060248140675957;1" dur="1s" repeatCount="indefinite" begin="-0.9257599774553938s"></animate>
</circle><circle cx="29" cy="122.37303701766326" r="22" fill="#f47e60">
<animate attributeName="cy" values="122.37303701766326;-17.181874655618834" keyTimes="0;1" dur="1s" repeatCount="indefinite" begin="-0.11979523584713825s"></animate>
<animate attributeName="r" values="22;0;0" keyTimes="0;0.5778892301319281;1" dur="1s" repeatCount="indefinite" begin="-0.11979523584713825s"></animate>
</circle><circle cx="30" cy="132.91741320840808" r="18" fill="#f47e60">
<animate attributeName="cy" values="132.91741320840808;0.24294121648419775" keyTimes="0;1" dur="1s" repeatCount="indefinite" begin="-0.6890213202603488s"></animate>
<animate attributeName="r" values="18;0;0" keyTimes="0;0.8587373770805918;1" dur="1s" repeatCount="indefinite" begin="-0.6890213202603488s"></animate>
</circle><circle cx="80" cy="116.72839679840811" r="14" fill="#f47e60">
<animate attributeName="cy" values="116.72839679840811;4.82183707831593" keyTimes="0;1" dur="1s" repeatCount="indefinite" begin="-0.08182847032405782s"></animate>
<animate attributeName="r" values="14;0;0" keyTimes="0;0.6809633164153448;1" dur="1s" repeatCount="indefinite" begin="-0.08182847032405782s"></animate>
</circle><circle cx="31" cy="125.20247260666616" r="13" fill="#f47e60">
<animate attributeName="cy" values="125.20247260666616;2.008326413572634" keyTimes="0;1" dur="1s" repeatCount="indefinite" begin="-0.8369662812852767s"></animate>
<animate attributeName="r" values="13;0;0" keyTimes="0;0.5845779670186058;1" dur="1s" repeatCount="indefinite" begin="-0.8369662812852767s"></animate>
</circle><circle cx="60" cy="125.0794549947879" r="16" fill="#f47e60">
<animate attributeName="cy" values="125.0794549947879;0.7338248372355807" keyTimes="0;1" dur="1s" repeatCount="indefinite" begin="-0.8948237868324189s"></animate>
<animate attributeName="r" values="16;0;0" keyTimes="0;0.9120596722058173;1" dur="1s" repeatCount="indefinite" begin="-0.8948237868324189s"></animate>
</circle><circle cx="25" cy="126.90612837175388" r="8" fill="#f47e60">
<animate attributeName="cy" values="126.90612837175388;4.0472618983783715" keyTimes="0;1" dur="1s" repeatCount="indefinite" begin="-0.39581604043317986s"></animate>
<animate attributeName="r" values="8;0;0" keyTimes="0;0.8074064845720312;1" dur="1s" repeatCount="indefinite" begin="-0.39581604043317986s"></animate>
</circle><circle cx="37" cy="131.42028038990128" r="25" fill="#f47e60">
<animate attributeName="cy" values="131.42028038990128;-22.403977227715075" keyTimes="0;1" dur="1s" repeatCount="indefinite" begin="-0.04301794169924622s"></animate>
<animate attributeName="r" values="25;0;0" keyTimes="0;0.524891315929541;1" dur="1s" repeatCount="indefinite" begin="-0.04301794169924622s"></animate>
</circle><circle cx="41" cy="149.05000141391616" r="31" fill="#f47e60">
<animate attributeName="cy" values="149.05000141391616;-19.10046896539864" keyTimes="0;1" dur="1s" repeatCount="indefinite" begin="-0.7213401886638007s"></animate>
<animate attributeName="r" values="31;0;0" keyTimes="0;0.6890520162965066;1" dur="1s" repeatCount="indefinite" begin="-0.7213401886638007s"></animate>
</circle><circle cx="36" cy="138.58798523568342" r="27" fill="#f47e60">
<animate attributeName="cy" values="138.58798523568342;-15.572058043829461" keyTimes="0;1" dur="1s" repeatCount="indefinite" begin="-0.40556498158772736s"></animate>
<animate attributeName="r" values="27;0;0" keyTimes="0;0.8506348676044777;1" dur="1s" repeatCount="indefinite" begin="-0.40556498158772736s"></animate>
</circle><circle cx="78" cy="137.9707233461312" r="20" fill="#f47e60">
<animate attributeName="cy" values="137.9707233461312;-3.6945948738885512" keyTimes="0;1" dur="1s" repeatCount="indefinite" begin="-0.8880631706610672s"></animate>
<animate attributeName="r" values="20;0;0" keyTimes="0;0.9304971995517395;1" dur="1s" repeatCount="indefinite" begin="-0.8880631706610672s"></animate>
</circle><circle cx="79" cy="134.71673525431498" r="18" fill="#f47e60">
<animate attributeName="cy" values="134.71673525431498;-10.261412982322742" keyTimes="0;1" dur="1s" repeatCount="indefinite" begin="-0.2848983056723242s"></animate>
<animate attributeName="r" values="18;0;0" keyTimes="0;0.7526875949615255;1" dur="1s" repeatCount="indefinite" begin="-0.2848983056723242s"></animate>
</circle><circle cx="82" cy="111.49802891873294" r="5" fill="#f47e60">
<animate attributeName="cy" values="111.49802891873294;12.140748225430922" keyTimes="0;1" dur="1s" repeatCount="indefinite" begin="-0.40945179236345397s"></animate>
<animate attributeName="r" values="5;0;0" keyTimes="0;0.703997116139137;1" dur="1s" repeatCount="indefinite" begin="-0.40945179236345397s"></animate>
</circle><circle cx="68" cy="140.96466884045572" r="22" fill="#f47e60">
<animate attributeName="cy" values="140.96466884045572;-4.079142984351218" keyTimes="0;1" dur="1s" repeatCount="indefinite" begin="-0.40439383112303107s"></animate>
<animate attributeName="r" values="22;0;0" keyTimes="0;0.5493704483007363;1" dur="1s" repeatCount="indefinite" begin="-0.40439383112303107s"></animate>
</circle><circle cx="41" cy="116.24169615516264" r="16" fill="#f47e60">
<animate attributeName="cy" values="116.24169615516264;-13.644720096932094" keyTimes="0;1" dur="1s" repeatCount="indefinite" begin="-0.22449184929827926s"></animate>
<animate attributeName="r" values="16;0;0" keyTimes="0;0.6587866247823291;1" dur="1s" repeatCount="indefinite" begin="-0.22449184929827926s"></animate>
</circle><circle cx="20" cy="124.66929057881916" r="15" fill="#f47e60">
<animate attributeName="cy" values="124.66929057881916;2.5505611618972814" keyTimes="0;1" dur="1s" repeatCount="indefinite" begin="-0.017560126563357925s"></animate>
<animate attributeName="r" values="15;0;0" keyTimes="0;0.6128429739262174;1" dur="1s" repeatCount="indefinite" begin="-0.017560126563357925s"></animate>
</circle><circle cx="63" cy="126.5115900704738" r="26" fill="#f47e60">
<animate attributeName="cy" values="126.5115900704738;-20.921901271813873" keyTimes="0;1" dur="1s" repeatCount="indefinite" begin="-0.5285257319858678s"></animate>
<animate attributeName="r" values="26;0;0" keyTimes="0;0.9007468611639214;1" dur="1s" repeatCount="indefinite" begin="-0.5285257319858678s"></animate>
</circle><circle cx="90" cy="111.61440083571019" r="6" fill="#f47e60">
<animate attributeName="cy" values="111.61440083571019;11.61930520437923" keyTimes="0;1" dur="1s" repeatCount="indefinite" begin="-0.8167452043810126s"></animate>
<animate attributeName="r" values="6;0;0" keyTimes="0;0.9810779841180124;1" dur="1s" repeatCount="indefinite" begin="-0.8167452043810126s"></animate>
</circle><circle cx="78" cy="122.50775060552778" r="20" fill="#f47e60">
<animate attributeName="cy" values="122.50775060552778;-4.59807973956865" keyTimes="0;1" dur="1s" repeatCount="indefinite" begin="-0.11755589684814727s"></animate>
<animate attributeName="r" values="20;0;0" keyTimes="0;0.6705237343698631;1" dur="1s" repeatCount="indefinite" begin="-0.11755589684814727s"></animate>
</circle><circle cx="31" cy="127.90703241028092" r="9" fill="#f47e60">
<animate attributeName="cy" values="127.90703241028092;0.829718008041219" keyTimes="0;1" dur="1s" repeatCount="indefinite" begin="-0.5851309189776632s"></animate>
<animate attributeName="r" values="9;0;0" keyTimes="0;0.6889560303799027;1" dur="1s" repeatCount="indefinite" begin="-0.5851309189776632s"></animate>
</circle><circle cx="65" cy="117.43435709704966" r="4" fill="#f47e60">
<animate attributeName="cy" values="117.43435709704966;15.28596080488979" keyTimes="0;1" dur="1s" repeatCount="indefinite" begin="-0.8492165554334472s"></animate>
<animate attributeName="r" values="4;0;0" keyTimes="0;0.5287459347086204;1" dur="1s" repeatCount="indefinite" begin="-0.8492165554334472s"></animate>
</circle><circle cx="89" cy="122.93132420091489" r="3" fill="#f47e60">
<animate attributeName="cy" values="122.93132420091489;5.980513428860888" keyTimes="0;1" dur="1s" repeatCount="indefinite" begin="-0.06884209677796871s"></animate>
<animate attributeName="r" values="3;0;0" keyTimes="0;0.5868616814040618;1" dur="1s" repeatCount="indefinite" begin="-0.06884209677796871s"></animate>
</circle><circle cx="68" cy="129.1441504106191" r="26" fill="#f47e60">
<animate attributeName="cy" values="129.1441504106191;-22.781245889673905" keyTimes="0;1" dur="1s" repeatCount="indefinite" begin="-0.26191875209122073s"></animate>
<animate attributeName="r" values="26;0;0" keyTimes="0;0.6200648439404779;1" dur="1s" repeatCount="indefinite" begin="-0.26191875209122073s"></animate>
</circle><circle cx="22" cy="130.63745849588264" r="20" fill="#f47e60">
<animate attributeName="cy" values="130.63745849588264;-10.695329441338862" keyTimes="0;1" dur="1s" repeatCount="indefinite" begin="-0.6192951915425052s"></animate>
<animate attributeName="r" values="20;0;0" keyTimes="0;0.6969346125529845;1" dur="1s" repeatCount="indefinite" begin="-0.6192951915425052s"></animate>
</circle></g><g filter="url(#ldio-ekpf7uvh2aq-filter)"><circle cx="57" cy="123.68953191890479" r="12" fill="#f8b26a">
<animate attributeName="cy" values="123.68953191890479;4.854991577389438" keyTimes="0;1" dur="1s" repeatCount="indefinite" begin="-0.9097135632734302s"></animate>
<animate attributeName="r" values="12;0;0" keyTimes="0;0.9463910575266388;1" dur="1s" repeatCount="indefinite" begin="-0.9097135632734302s"></animate>
</circle><circle cx="24" cy="124.54645838615471" r="12" fill="#f8b26a">
<animate attributeName="cy" values="124.54645838615471;-11.813810322332547" keyTimes="0;1" dur="1s" repeatCount="indefinite" begin="-0.007050694143823311s"></animate>
<animate attributeName="r" values="12;0;0" keyTimes="0;0.7078891674964196;1" dur="1s" repeatCount="indefinite" begin="-0.007050694143823311s"></animate>
</circle><circle cx="54" cy="110.08044357995595" r="3" fill="#f8b26a">
<animate attributeName="cy" values="110.08044357995595;13.402947007936334" keyTimes="0;1" dur="1s" repeatCount="indefinite" begin="-0.994432759852213s"></animate>
<animate attributeName="r" values="3;0;0" keyTimes="0;0.8430605754104277;1" dur="1s" repeatCount="indefinite" begin="-0.994432759852213s"></animate>
</circle><circle cx="49" cy="127.80477114160061" r="16" fill="#f8b26a">
<animate attributeName="cy" values="127.80477114160061;2.7658256519770603" keyTimes="0;1" dur="1s" repeatCount="indefinite" begin="-0.07188593356616135s"></animate>
<animate attributeName="r" values="16;0;0" keyTimes="0;0.6049768163612267;1" dur="1s" repeatCount="indefinite" begin="-0.07188593356616135s"></animate>
</circle><circle cx="52" cy="112.09746694041411" r="10" fill="#f8b26a">
<animate attributeName="cy" values="112.09746694041411;-2.8104821907767574" keyTimes="0;1" dur="1s" repeatCount="indefinite" begin="-0.4132445270517203s"></animate>
<animate attributeName="r" values="10;0;0" keyTimes="0;0.7843188648425736;1" dur="1s" repeatCount="indefinite" begin="-0.4132445270517203s"></animate>
</circle><circle cx="68" cy="119.76797510227266" r="15" fill="#f8b26a">
<animate attributeName="cy" values="119.76797510227266;-2.3187957684067317" keyTimes="0;1" dur="1s" repeatCount="indefinite" begin="-0.6317748306797277s"></animate>
<animate attributeName="r" values="15;0;0" keyTimes="0;0.8464277838946668;1" dur="1s" repeatCount="indefinite" begin="-0.6317748306797277s"></animate>
</circle><circle cx="17" cy="121.7997527406382" r="5" fill="#f8b26a">
<animate attributeName="cy" values="121.7997527406382;13.556957891026624" keyTimes="0;1" dur="1s" repeatCount="indefinite" begin="-0.9136732084136533s"></animate>
<animate attributeName="r" values="5;0;0" keyTimes="0;0.5349721785314134;1" dur="1s" repeatCount="indefinite" begin="-0.9136732084136533s"></animate>
</circle><circle cx="59" cy="116.30296558149124" r="4" fill="#f8b26a">
<animate attributeName="cy" values="116.30296558149124;-1.0433564145924477" keyTimes="0;1" dur="1s" repeatCount="indefinite" begin="-0.08891813207741484s"></animate>
<animate attributeName="r" values="4;0;0" keyTimes="0;0.6574981312374213;1" dur="1s" repeatCount="indefinite" begin="-0.08891813207741484s"></animate>
</circle><circle cx="88" cy="113.1583378513422" r="12" fill="#f8b26a">
<animate attributeName="cy" values="113.1583378513422;1.456869512308952" keyTimes="0;1" dur="1s" repeatCount="indefinite" begin="-0.14992898603700067s"></animate>
<animate attributeName="r" values="12;0;0" keyTimes="0;0.9565108058771807;1" dur="1s" repeatCount="indefinite" begin="-0.14992898603700067s"></animate>
</circle><circle cx="84" cy="112.41279273844411" r="10" fill="#f8b26a">
<animate attributeName="cy" values="112.41279273844411;1.6491176590177243" keyTimes="0;1" dur="1s" repeatCount="indefinite" begin="-0.5833010262862421s"></animate>
<animate attributeName="r" values="10;0;0" keyTimes="0;0.5438806242531744;1" dur="1s" repeatCount="indefinite" begin="-0.5833010262862421s"></animate>
</circle><circle cx="87" cy="120.26530337145327" r="5" fill="#f8b26a">
<animate attributeName="cy" values="120.26530337145327;9.388664939149207" keyTimes="0;1" dur="1s" repeatCount="indefinite" begin="-0.05018189342538548s"></animate>
<animate attributeName="r" values="5;0;0" keyTimes="0;0.637897648645736;1" dur="1s" repeatCount="indefinite" begin="-0.05018189342538548s"></animate>
</circle><circle cx="24" cy="123.99448894779877" r="9" fill="#f8b26a">
<animate attributeName="cy" values="123.99448894779877;2.3750067806866078" keyTimes="0;1" dur="1s" repeatCount="indefinite" begin="-0.8890495329191316s"></animate>
<animate attributeName="r" values="9;0;0" keyTimes="0;0.663064102718458;1" dur="1s" repeatCount="indefinite" begin="-0.8890495329191316s"></animate>
</circle><circle cx="73" cy="120.00019528994846" r="12" fill="#f8b26a">
<animate attributeName="cy" values="120.00019528994846;-9.503507375076166" keyTimes="0;1" dur="1s" repeatCount="indefinite" begin="-0.6351313241419324s"></animate>
<animate attributeName="r" values="12;0;0" keyTimes="0;0.9354194941922095;1" dur="1s" repeatCount="indefinite" begin="-0.6351313241419324s"></animate>
</circle><circle cx="74" cy="113.88820186698781" r="4" fill="#f8b26a">
<animate attributeName="cy" values="113.88820186698781;10.570535200732685" keyTimes="0;1" dur="1s" repeatCount="indefinite" begin="-0.7132998998028989s"></animate>
<animate attributeName="r" values="4;0;0" keyTimes="0;0.91895021859856;1" dur="1s" repeatCount="indefinite" begin="-0.7132998998028989s"></animate>
</circle><circle cx="68" cy="129.5841522641359" r="12" fill="#f8b26a">
<animate attributeName="cy" values="129.5841522641359;3.894919008898638" keyTimes="0;1" dur="1s" repeatCount="indefinite" begin="-0.29330391921510546s"></animate>
<animate attributeName="r" values="12;0;0" keyTimes="0;0.9096568793749455;1" dur="1s" repeatCount="indefinite" begin="-0.29330391921510546s"></animate>
</circle><circle cx="53" cy="119.31720358172306" r="9" fill="#f8b26a">
<animate attributeName="cy" values="119.31720358172306;9.73624644875764" keyTimes="0;1" dur="1s" repeatCount="indefinite" begin="-0.9958245939061628s"></animate>
<animate attributeName="r" values="9;0;0" keyTimes="0;0.8571965277158554;1" dur="1s" repeatCount="indefinite" begin="-0.9958245939061628s"></animate>
</circle><circle cx="76" cy="134.80739606982607" r="17" fill="#f8b26a">
<animate attributeName="cy" values="134.80739606982607;0.3932385595869441" keyTimes="0;1" dur="1s" repeatCount="indefinite" begin="-0.8607153243461125s"></animate>
<animate attributeName="r" values="17;0;0" keyTimes="0;0.8654455107706405;1" dur="1s" repeatCount="indefinite" begin="-0.8607153243461125s"></animate>
</circle><circle cx="75" cy="122.61568996754474" r="7" fill="#f8b26a">
<animate attributeName="cy" values="122.61568996754474;10.652526875734779" keyTimes="0;1" dur="1s" repeatCount="indefinite" begin="-0.959721298983397s"></animate>
<animate attributeName="r" values="7;0;0" keyTimes="0;0.6271803990132601;1" dur="1s" repeatCount="indefinite" begin="-0.959721298983397s"></animate>
</circle><circle cx="87" cy="115.0788054109218" r="12" fill="#f8b26a">
<animate attributeName="cy" values="115.0788054109218;-8.15567938666852" keyTimes="0;1" dur="1s" repeatCount="indefinite" begin="-0.0690058777440068s"></animate>
<animate attributeName="r" values="12;0;0" keyTimes="0;0.6627211388649489;1" dur="1s" repeatCount="indefinite" begin="-0.0690058777440068s"></animate>
</circle><circle cx="21" cy="118.08738171978098" r="9" fill="#f8b26a">
<animate attributeName="cy" values="118.08738171978098;-4.9475469075625504" keyTimes="0;1" dur="1s" repeatCount="indefinite" begin="-0.7078831683260647s"></animate>
<animate attributeName="r" values="9;0;0" keyTimes="0;0.9501044367725069;1" dur="1s" repeatCount="indefinite" begin="-0.7078831683260647s"></animate>
</circle><circle cx="24" cy="128.09150085659442" r="9" fill="#f8b26a">
<animate attributeName="cy" values="128.09150085659442;2.7320353690265122" keyTimes="0;1" dur="1s" repeatCount="indefinite" begin="-0.521121701341132s"></animate>
<animate attributeName="r" values="9;0;0" keyTimes="0;0.7357531229285373;1" dur="1s" repeatCount="indefinite" begin="-0.521121701341132s"></animate>
</circle><circle cx="26" cy="127.49368345428452" r="15" fill="#f8b26a">
<animate attributeName="cy" values="127.49368345428452;-10.361246269666196" keyTimes="0;1" dur="1s" repeatCount="indefinite" begin="-0.9420307783603239s"></animate>
<animate attributeName="r" values="15;0;0" keyTimes="0;0.7467409545014994;1" dur="1s" repeatCount="indefinite" begin="-0.9420307783603239s"></animate>
</circle><circle cx="39" cy="114.20744515306558" r="6" fill="#f8b26a">
<animate attributeName="cy" values="114.20744515306558;5.606516894440285" keyTimes="0;1" dur="1s" repeatCount="indefinite" begin="-0.49268347147689695s"></animate>
<animate attributeName="r" values="6;0;0" keyTimes="0;0.5874854761603912;1" dur="1s" repeatCount="indefinite" begin="-0.49268347147689695s"></animate>
</circle><circle cx="61" cy="123.10463246179438" r="11" fill="#f8b26a">
<animate attributeName="cy" values="123.10463246179438;-5.189366828773049" keyTimes="0;1" dur="1s" repeatCount="indefinite" begin="-0.21359109324800063s"></animate>
<animate attributeName="r" values="11;0;0" keyTimes="0;0.6970744691674484;1" dur="1s" repeatCount="indefinite" begin="-0.21359109324800063s"></animate>
</circle><circle cx="37" cy="115.40335155247101" r="10" fill="#f8b26a">
<animate attributeName="cy" values="115.40335155247101;3.4285850566842946" keyTimes="0;1" dur="1s" repeatCount="indefinite" begin="-0.5344545499798534s"></animate>
<animate attributeName="r" values="10;0;0" keyTimes="0;0.9983685792824288;1" dur="1s" repeatCount="indefinite" begin="-0.5344545499798534s"></animate>
</circle><circle cx="22" cy="124.59228223795324" r="7" fill="#f8b26a">
<animate attributeName="cy" values="124.59228223795324;-3.5076355130396912" keyTimes="0;1" dur="1s" repeatCount="indefinite" begin="-0.8102510016775601s"></animate>
<animate attributeName="r" values="7;0;0" keyTimes="0;0.6369981578428732;1" dur="1s" repeatCount="indefinite" begin="-0.8102510016775601s"></animate>
</circle><circle cx="34" cy="111.69621652751701" r="5" fill="#f8b26a">
<animate attributeName="cy" values="111.69621652751701;13.965538669421832" keyTimes="0;1" dur="1s" repeatCount="indefinite" begin="-0.3819120829819431s"></animate>
<animate attributeName="r" values="5;0;0" keyTimes="0;0.9240036927970401;1" dur="1s" repeatCount="indefinite" begin="-0.3819120829819431s"></animate>
</circle><circle cx="61" cy="121.99207528226256" r="6" fill="#f8b26a">
<animate attributeName="cy" values="121.99207528226256;-1.1884130816048284" keyTimes="0;1" dur="1s" repeatCount="indefinite" begin="-0.351012424136126s"></animate>
<animate attributeName="r" values="6;0;0" keyTimes="0;0.9527855705617168;1" dur="1s" repeatCount="indefinite" begin="-0.351012424136126s"></animate>
</circle><circle cx="32" cy="115.36386365084275" r="13" fill="#f8b26a">
<animate attributeName="cy" values="115.36386365084275;-7.635796261623495" keyTimes="0;1" dur="1s" repeatCount="indefinite" begin="-0.22026693987990997s"></animate>
<animate attributeName="r" values="13;0;0" keyTimes="0;0.6822821982216503;1" dur="1s" repeatCount="indefinite" begin="-0.22026693987990997s"></animate>
</circle><circle cx="38" cy="123.93260454500944" r="10" fill="#f8b26a">
<animate attributeName="cy" values="123.93260454500944;-9.019646946232784" keyTimes="0;1" dur="1s" repeatCount="indefinite" begin="-0.5897767052001425s"></animate>
<animate attributeName="r" values="10;0;0" keyTimes="0;0.747643174639248;1" dur="1s" repeatCount="indefinite" begin="-0.5897767052001425s"></animate>
</circle><circle cx="91" cy="111.20360670124936" r="4" fill="#f8b26a">
<animate attributeName="cy" values="111.20360670124936;-2.7511383786778185" keyTimes="0;1" dur="1s" repeatCount="indefinite" begin="-0.5936715943771124s"></animate>
<animate attributeName="r" values="4;0;0" keyTimes="0;0.5292863982274825;1" dur="1s" repeatCount="indefinite" begin="-0.5936715943771124s"></animate>
</circle><circle cx="93" cy="109.08688866758263" r="6" fill="#f8b26a">
<animate attributeName="cy" values="109.08688866758263;13.986514639855155" keyTimes="0;1" dur="1s" repeatCount="indefinite" begin="-0.20182465253134418s"></animate>
<animate attributeName="r" values="6;0;0" keyTimes="0;0.9578727930035874;1" dur="1s" repeatCount="indefinite" begin="-0.20182465253134418s"></animate>
</circle><circle cx="90" cy="115.44258946143852" r="3" fill="#f8b26a">
<animate attributeName="cy" values="115.44258946143852;7.971557449807172" keyTimes="0;1" dur="1s" repeatCount="indefinite" begin="-0.8138344996352406s"></animate>
<animate attributeName="r" values="3;0;0" keyTimes="0;0.822677504532275;1" dur="1s" repeatCount="indefinite" begin="-0.8138344996352406s"></animate>
</circle><circle cx="24" cy="130.98782632438636" r="15" fill="#f8b26a">
<animate attributeName="cy" values="130.98782632438636;-11.868426017755008" keyTimes="0;1" dur="1s" repeatCount="indefinite" begin="-0.8574009914089539s"></animate>
<animate attributeName="r" values="15;0;0" keyTimes="0;0.8610318085552064;1" dur="1s" repeatCount="indefinite" begin="-0.8574009914089539s"></animate>
</circle><circle cx="49" cy="122.24309971563434" r="14" fill="#f8b26a">
<animate attributeName="cy" values="122.24309971563434;3.5685994935617273" keyTimes="0;1" dur="1s" repeatCount="indefinite" begin="-0.4267384904796552s"></animate>
<animate attributeName="r" values="14;0;0" keyTimes="0;0.5503829186981541;1" dur="1s" repeatCount="indefinite" begin="-0.4267384904796552s"></animate>
</circle><circle cx="18" cy="117.38217971971676" r="9" fill="#f8b26a">
<animate attributeName="cy" values="117.38217971971676;6.631006164776416" keyTimes="0;1" dur="1s" repeatCount="indefinite" begin="-0.6828218424869835s"></animate>
<animate attributeName="r" values="9;0;0" keyTimes="0;0.6808177575913787;1" dur="1s" repeatCount="indefinite" begin="-0.6828218424869835s"></animate>
</circle><circle cx="78" cy="124.28678852303256" r="15" fill="#f8b26a">
<animate attributeName="cy" values="124.28678852303256;1.3740946843405304" keyTimes="0;1" dur="1s" repeatCount="indefinite" begin="-0.4161035078940827s"></animate>
<animate attributeName="r" values="15;0;0" keyTimes="0;0.6388001474427218;1" dur="1s" repeatCount="indefinite" begin="-0.4161035078940827s"></animate>
</circle><circle cx="44" cy="106.6189204965897" r="3" fill="#f8b26a">
<animate attributeName="cy" values="106.6189204965897;16.750815514807034" keyTimes="0;1" dur="1s" repeatCount="indefinite" begin="-0.0510803765953457s"></animate>
<animate attributeName="r" values="3;0;0" keyTimes="0;0.7907276882734477;1" dur="1s" repeatCount="indefinite" begin="-0.0510803765953457s"></animate>
</circle><circle cx="41" cy="119.64799537397232" r="5" fill="#f8b26a">
<animate attributeName="cy" values="119.64799537397232;6.398667601394809" keyTimes="0;1" dur="1s" repeatCount="indefinite" begin="-0.4280945050279754s"></animate>
<animate attributeName="r" values="5;0;0" keyTimes="0;0.5751942250658201;1" dur="1s" repeatCount="indefinite" begin="-0.4280945050279754s"></animate>
</circle><circle cx="19" cy="120.0916729802829" r="10" fill="#f8b26a">
<animate attributeName="cy" values="120.0916729802829;-9.513704965243033" keyTimes="0;1" dur="1s" repeatCount="indefinite" begin="-0.043405970368113445s"></animate>
<animate attributeName="r" values="10;0;0" keyTimes="0;0.5435267537060107;1" dur="1s" repeatCount="indefinite" begin="-0.043405970368113445s"></animate>
</circle><circle cx="61" cy="123.62714133794762" r="5" fill="#f8b26a">
<animate attributeName="cy" values="123.62714133794762;2.362315551662477" keyTimes="0;1" dur="1s" repeatCount="indefinite" begin="-0.5256540407430482s"></animate>
<animate attributeName="r" values="5;0;0" keyTimes="0;0.9222037100732456;1" dur="1s" repeatCount="indefinite" begin="-0.5256540407430482s"></animate>
</circle><circle cx="64" cy="115.25525614926073" r="13" fill="#f8b26a">
<animate attributeName="cy" values="115.25525614926073;-10.304511881341815" keyTimes="0;1" dur="1s" repeatCount="indefinite" begin="-0.6633519944592159s"></animate>
<animate attributeName="r" values="13;0;0" keyTimes="0;0.5401283508859178;1" dur="1s" repeatCount="indefinite" begin="-0.6633519944592159s"></animate>
</circle><circle cx="12" cy="129.13660549492693" r="11" fill="#f8b26a">
<animate attributeName="cy" values="129.13660549492693;-7.965594883525825" keyTimes="0;1" dur="1s" repeatCount="indefinite" begin="-0.9929282227674491s"></animate>
<animate attributeName="r" values="11;0;0" keyTimes="0;0.9536114994321867;1" dur="1s" repeatCount="indefinite" begin="-0.9929282227674491s"></animate>
</circle><circle cx="39" cy="106.95504126040025" r="2" fill="#f8b26a">
<animate attributeName="cy" values="106.95504126040025;5.834416891524681" keyTimes="0;1" dur="1s" repeatCount="indefinite" begin="-0.22005892301327157s"></animate>
<animate attributeName="r" values="2;0;0" keyTimes="0;0.6089960643653531;1" dur="1s" repeatCount="indefinite" begin="-0.22005892301327157s"></animate>
</circle><circle cx="30" cy="112.12744151244388" r="8" fill="#f8b26a">
<animate attributeName="cy" values="112.12744151244388;-4.465606537168944" keyTimes="0;1" dur="1s" repeatCount="indefinite" begin="-0.24710322548242414s"></animate>
<animate attributeName="r" values="8;0;0" keyTimes="0;0.7479705418636007;1" dur="1s" repeatCount="indefinite" begin="-0.24710322548242414s"></animate>
</circle><circle cx="67" cy="124.83294711941956" r="16" fill="#f8b26a">
<animate attributeName="cy" values="124.83294711941956;-7.6291463245052284" keyTimes="0;1" dur="1s" repeatCount="indefinite" begin="-0.614066023590482s"></animate>
<animate attributeName="r" values="16;0;0" keyTimes="0;0.7584434636145084;1" dur="1s" repeatCount="indefinite" begin="-0.614066023590482s"></animate>
</circle><circle cx="22" cy="119.36463088979876" r="4" fill="#f8b26a">
<animate attributeName="cy" values="119.36463088979876;12.12664234343379" keyTimes="0;1" dur="1s" repeatCount="indefinite" begin="-0.527385385953813s"></animate>
<animate attributeName="r" values="4;0;0" keyTimes="0;0.5661680148267347;1" dur="1s" repeatCount="indefinite" begin="-0.527385385953813s"></animate>
</circle><circle cx="12" cy="122.52124979151506" r="7" fill="#f8b26a">
<animate attributeName="cy" values="122.52124979151506;3.7506712743784085" keyTimes="0;1" dur="1s" repeatCount="indefinite" begin="-0.37225883133903837s"></animate>
<animate attributeName="r" values="7;0;0" keyTimes="0;0.9003327357718601;1" dur="1s" repeatCount="indefinite" begin="-0.37225883133903837s"></animate>
</circle><circle cx="69" cy="130.5210986475815" r="14" fill="#f8b26a">
<animate attributeName="cy" values="130.5210986475815;-0.30973651460238827" keyTimes="0;1" dur="1s" repeatCount="indefinite" begin="-0.6062299863585278s"></animate>
<animate attributeName="r" values="14;0;0" keyTimes="0;0.9220180768904789;1" dur="1s" repeatCount="indefinite" begin="-0.6062299863585278s"></animate>
</circle><circle cx="20" cy="114.80243604193255" r="9" fill="#f8b26a">
<animate attributeName="cy" values="114.80243604193255;7.19374553530416" keyTimes="0;1" dur="1s" repeatCount="indefinite" begin="-0.6866227460985781s"></animate>
<animate attributeName="r" values="9;0;0" keyTimes="0;0.6690048284116141;1" dur="1s" repeatCount="indefinite" begin="-0.6866227460985781s"></animate>
</circle></g>
</svg>
Side by Side

After

Width:  |  Height:  |  Size: 58 KiB

BIN
assets/test.pdf Normal file
View File

Binary file not shown.

BIN
assets/web_demo.gif Normal file
View File

Binary file not shown.
Side by Side

After

Width:  |  Height:  |  Size: 10 MiB

12
requirements.txt Normal file
View File

@@ -0,0 +1,12 @@
transformers
datasets
evaluate
streamlit
opencv-python
ray[serve]
accelerate
tensorboardX
nltk
python-multipart
pdf2image

10
src/client_demo.py Normal file
View File

@@ -0,0 +1,10 @@
import requests
url = "http://127.0.0.1:8000/predict"
img_path = "/your/image/path/"
with open(img_path, 'rb') as img:
files = {'img': img}
response = requests.post(url, files=files)
print(response.text)

36
src/inference.py Normal file
View File

@@ -0,0 +1,36 @@
import os
import argparse
from pathlib import Path
from models.ocr_model.utils.inference import inference
from models.ocr_model.model.TexTeller import TexTeller
if __name__ == '__main__':
os.chdir(Path(__file__).resolve().parent)
parser = argparse.ArgumentParser()
parser.add_argument(
'-img',
type=str,
required=True,
help='path to the input image'
)
parser.add_argument(
'-cuda',
default=False,
action='store_true',
help='use cuda or not'
)
args = parser.parse_args()
# You can use your own checkpoint and tokenizer path.
print('Loading model and tokenizer...')
model = TexTeller.from_pretrained()
tokenizer = TexTeller.get_tokenizer()
print('Model and tokenizer loaded.')
img_path = [args.img]
print('Inference...')
res = inference(model, tokenizer, img_path, args.cuda)
print(res[0])

23
src/models/globals.py Normal file
View File

@@ -0,0 +1,23 @@
# Formula image(grayscale) mean and variance
IMAGE_MEAN = 0.9545467
IMAGE_STD = 0.15394445
# Vocabulary size for TexTeller
VOCAB_SIZE = 15000
# Fixed size for input image for TexTeller
FIXED_IMG_SIZE = 448
# Image channel for TexTeller
IMG_CHANNELS = 1 # grayscale image
# Max size of token for embedding
MAX_TOKEN_SIZE = 1024
# Scaling ratio for random resizing when training
MAX_RESIZE_RATIO = 1.15
MIN_RESIZE_RATIO = 0.75
# Minimum height and width for input image for TexTeller
MIN_HEIGHT = 12
MIN_WIDTH = 30

43
src/models/ocr_model/model/TexTeller.py Normal file
View File

@@ -0,0 +1,43 @@
from pathlib import Path
from models.globals import (
VOCAB_SIZE,
FIXED_IMG_SIZE,
IMG_CHANNELS,
)
from transformers import (
ViTConfig,
ViTModel,
TrOCRConfig,
TrOCRForCausalLM,
RobertaTokenizerFast,
VisionEncoderDecoderModel,
)
class TexTeller(VisionEncoderDecoderModel):
REPO_NAME = '/home/lhy/code/TexTeller/src/models/ocr_model/train/train_result/TexTellerv2/checkpoint-588000'
def __init__(self, decoder_path=None, tokenizer_path=None):
encoder = ViTModel(ViTConfig(
image_size=FIXED_IMG_SIZE,
num_channels=IMG_CHANNELS
))
decoder = TrOCRForCausalLM(TrOCRConfig(
vocab_size=VOCAB_SIZE,
))
super().__init__(encoder=encoder, decoder=decoder)
@classmethod
def from_pretrained(cls, model_path: str = None):
if model_path is None or model_path == 'default':
return VisionEncoderDecoderModel.from_pretrained(cls.REPO_NAME)
model_path = Path(model_path).resolve()
return VisionEncoderDecoderModel.from_pretrained(str(model_path))
@classmethod
def get_tokenizer(cls, tokenizer_path: str = None) -> RobertaTokenizerFast:
if tokenizer_path is None or tokenizer_path == 'default':
return RobertaTokenizerFast.from_pretrained(cls.REPO_NAME)
tokenizer_path = Path(tokenizer_path).resolve()
return RobertaTokenizerFast.from_pretrained(str(tokenizer_path))

35
src/models/ocr_model/train/dataset/formulas.jsonl Normal file
View File

@@ -0,0 +1,35 @@
{"img_name": "0.png", "formula": "\\[\\mathbb{C}^{4}\\stackrel{{\\pi_{1}}}{{\\longleftarrow}}\\mathcal{ F}\\stackrel{{\\pi_{2}}}{{\\rightarrow}}\\mathcal{PT},\\]"}
{"img_name": "1.png", "formula": "\\[W^{*}_{Z}(x_{1},x_{2})=W_{f\\lrcorner Z}(y_{1},y_{2})=\\mathcal{P}\\exp\\left( \\int_{\\gamma}A_{\\mu}dx^{\\mu}\\right).\\]"}
{"img_name": "2.png", "formula": "\\[G=W^{*}_{Z}(q,p)=\\tilde{H}H^{-1}\\]"}
{"img_name": "3.png", "formula": "\\[H=W^{*}_{Z}(p,x),\\ \\ \\tilde{H}=W^{*}_{Z}(q,x).\\]"}
{"img_name": "4.png", "formula": "\\[v\\cdot f^{*}A|_{x}=(f\\lrcorner Z)_{*}v\\cdot A|_{f\\lrcorner Z(x)},\\quad x\\in Z, \\ v\\in T_{x}Z.\\]"}
{"img_name": "5.png", "formula": "\\[(f\\lrcorner Z)_{*}v\\cdot A|_{f\\lrcorner Z(x)}=v^{\\alpha\\dot{\\alpha}}\\Big{(} \\frac{\\partial y^{\\beta\\dot{\\beta}}}{\\partial x^{\\alpha\\dot{\\alpha}}}A_{\\beta \\dot{\\beta}}\\Big{)}\\Big{|}_{f\\lrcorner Z(x)},\\ x\\in Z,\\ v\\in T_{x}Z,\\]"}
{"img_name": "6.png", "formula": "\\[\\{T_{i},T_{j}\\}=\\{\\tilde{T}^{i},\\tilde{T}^{j}\\}=0,\\ \\ \\{T_{i},\\tilde{T}^{j}\\}=2i \\delta^{j}_{i}D,\\]"}
{"img_name": "7.png", "formula": "\\[(\\partial_{s},q_{i},\\tilde{q}^{k})\\rightarrow(D,M^{j}_{i}T_{j},\\tilde{M}^{k}_ {l}\\tilde{T}^{l}),\\]"}
{"img_name": "8.png", "formula": "\\[M^{i}_{j}\\tilde{M}^{j}_{k}=\\delta^{i}_{k}.\\]"}
{"img_name": "9.png", "formula": "\\[Q_{i\\alpha}=q_{i\\alpha}+\\omega_{i\\alpha},\\ \\tilde{Q}^{i}_{\\dot{\\alpha}}=q^{i}_{ \\dot{\\alpha}}+\\tilde{\\omega}^{i}_{\\dot{\\alpha}},\\ D_{\\alpha\\dot{\\alpha}}= \\partial_{\\alpha\\dot{\\alpha}}+A_{\\alpha\\dot{\\alpha}}.\\]"}
{"img_name": "10.png", "formula": "\\[\\hat{f}(g,\\theta^{i\\alpha},\\tilde{\\theta}^{\\dot{\\alpha}}_{j})=(f(g),[V^{-1}]^ {\\alpha}_{\\beta}\\theta^{i\\beta},[\\tilde{V}^{-1}]^{\\dot{\\alpha}}_{\\dot{\\beta}} \\tilde{\\theta}^{\\dot{\\beta}}_{j}),\\ g\\in{\\cal G},\\]"}
{"img_name": "11.png", "formula": "\\[v^{\\beta\\dot{\\beta}}V^{\\alpha}_{\\beta}\\tilde{V}^{\\dot{\\alpha}}_{\\dot{\\beta}} =((f\\lrcorner L_{0})_{*}v)^{\\alpha\\dot{\\alpha}},\\]"}
{"img_name": "12.png", "formula": "\\[\\omega_{i\\alpha}=\\tilde{\\theta}^{\\dot{\\alpha}}_{i}h_{\\alpha\\dot{\\alpha}}(x^{ \\beta\\dot{\\beta}},\\tau^{\\beta\\dot{\\beta}}),\\ \\ \\tilde{\\omega}^{i}_{\\alpha}=\\theta^{i\\alpha}\\tilde{h}_{\\alpha\\dot{\\alpha}}(x^{ \\beta\\dot{\\beta}},\\tau^{\\beta\\dot{\\beta}}),\\]"}
{"img_name": "13.png", "formula": "\\[\\begin{split}&\\lambda^{\\alpha}\\hat{f}^{*}\\omega_{i\\alpha}(z)= \\tilde{\\theta}^{\\dot{\\beta}}_{i}\\lambda^{\\alpha}\\left(V^{\\beta}_{\\alpha}h_{ \\beta\\dot{\\beta}}(x^{\\prime},\\tau^{\\prime})\\right),\\\\ &\\tilde{\\lambda}^{\\dot{\\alpha}}\\hat{f}^{*}\\tilde{\\omega}^{i}_{ \\dot{\\alpha}}(z)=\\theta^{i\\beta}\\tilde{\\lambda}^{\\dot{\\alpha}}\\left(\\tilde{V}^ {\\dot{\\beta}}_{\\dot{\\alpha}}\\tilde{h}_{\\beta\\dot{\\beta}}(x^{\\prime},\\tau^{ \\prime})\\right),\\end{split}\\]"}
{"img_name": "14.png", "formula": "\\[A_{\\alpha\\dot{\\alpha}}=A_{\\alpha\\dot{\\alpha}}(x^{\\beta\\dot{\\beta}},\\tau^{ \\beta\\dot{\\beta}})\\]"}
{"img_name": "15.png", "formula": "\\[D=\\lambda^{\\alpha}\\tilde{\\lambda}^{\\dot{\\alpha}}D_{\\alpha\\dot{\\alpha}}\\]"}
{"img_name": "16.png", "formula": "\\[D=\\lambda^{\\alpha}\\tilde{\\lambda}^{\\dot{\\alpha}}\\partial_{\\alpha\\dot{\\alpha}}\\]"}
{"img_name": "17.png", "formula": "\\[[v_{1}\\cdot D^{*},v_{2}\\cdot D^{*}]=0\\]"}
{"img_name": "18.png", "formula": "\\[\\Phi_{A}=(\\omega_{i\\alpha},\\tilde{\\omega}^{i}_{\\dot{\\alpha}},A_{\\alpha\\dot{ \\alpha}})\\]"}
{"img_name": "19.png", "formula": "\\[\\hat{f}:{\\cal F}^{6|4N}\\rightarrow{\\cal F}^{6|4N}\\]"}
{"img_name": "20.png", "formula": "\\[\\sigma=(s,\\xi^{i},\\tilde{\\xi}_{j})\\in\\mathbb{C}^{1|2N}\\]"}
{"img_name": "21.png", "formula": "\\[\\tau^{\\alpha\\dot{\\alpha}}(h_{\\alpha\\dot{\\alpha}}+\\tilde{h}_{\\alpha\\dot{\\alpha} })=0\\]"}
{"img_name": "22.png", "formula": "\\[\\tau^{\\alpha\\dot{\\alpha}}\\rightarrow[V^{-1}]^{\\alpha}_{\\beta}[\\tilde{V}^{-1}]^{ \\dot{\\alpha}}_{\\dot{\\beta}}\\tau^{\\beta\\dot{\\beta}}\\]"}
{"img_name": "23.png", "formula": "\\[\\tau^{\\beta\\dot{\\beta}}=\\sum_{i}\\theta^{i\\beta}\\tilde{\\theta}^{\\dot{\\beta}}_{i}\\]"}
{"img_name": "24.png", "formula": "\\[\\theta^{i\\alpha}\\omega_{i\\alpha}+\\tilde{\\theta}^{i}_{\\dot{\\alpha}}\\tilde{ \\omega}^{\\dot{\\alpha}}_{i}=0\\]"}
{"img_name": "25.png", "formula": "\\[\\tilde{T}^{i}=\\tilde{\\lambda}^{\\dot{\\alpha}}\\tilde{Q}^{i}_{\\dot{\\alpha}}\\]"}
{"img_name": "26.png", "formula": "\\[\\tilde{T}^{i}=\\tilde{\\lambda}^{\\dot{\\alpha}}\\tilde{q}^{i}_{\\dot{\\alpha}}\\]"}
{"img_name": "27.png", "formula": "\\[\\tilde{\\lambda}^{\\dot{\\alpha}}f^{*}A_{\\alpha\\dot{\\alpha}}=H^{-1}\\tilde{ \\lambda}^{\\dot{\\alpha}}\\partial_{\\alpha\\dot{\\alpha}}H\\]"}
{"img_name": "28.png", "formula": "\\[\\tilde{q}^{i}=\\partial_{\\tilde{\\xi}_{i}}+i\\xi^{i}\\partial_{s}\\]"}
{"img_name": "29.png", "formula": "\\[\\tilde{q}^{i}_{\\dot{\\alpha}}=\\frac{\\partial}{\\partial\\tilde{\\theta}^{\\dot{ \\alpha}}_{i}}+i\\theta^{i\\alpha}\\frac{\\partial}{\\partial x^{\\alpha\\dot{\\alpha}}}\\]"}
{"img_name": "30.png", "formula": "\\[f\\lrcorner L(z)=\\pi_{1}\\circ f(z,\\lambda,\\tilde{\\lambda})\\ \\forall z\\in L\\]"}
{"img_name": "31.png", "formula": "\\[q_{i\\alpha}=\\frac{\\partial}{\\partial\\theta^{i\\alpha}}+i\\tilde{\\theta}^{\\dot{ \\alpha}}_{i}\\frac{\\partial}{\\partial x^{\\alpha\\dot{\\alpha}}}\\]"}
{"img_name": "32.png", "formula": "\\[q_{i}=\\partial_{\\xi^{i}}+i\\tilde{\\xi}_{i}\\partial_{s}\\]"}
{"img_name": "33.png", "formula": "\\[v^{\\alpha\\dot{\\alpha}}=\\lambda^{\\alpha}\\tilde{\\lambda}^{\\dot{\\alpha}}\\]"}
{"img_name": "34.png", "formula": "\\[z^{A}=(x^{\\alpha\\dot{\\alpha}},\\theta^{i\\alpha},\\tilde{\\theta}^{\\dot{\\alpha}}_{ j})\\]"}

BIN
src/models/ocr_model/train/dataset/images/0.png Normal file
View File

Binary file not shown.
Side by Side

After

Width:  |  Height:  |  Size: 3.1 KiB

BIN
src/models/ocr_model/train/dataset/images/1.png Normal file
View File

Binary file not shown.
Side by Side

After

Width:  |  Height:  |  Size: 8.7 KiB

BIN
src/models/ocr_model/train/dataset/images/10.png Normal file
View File

Binary file not shown.
Side by Side

After

Width:  |  Height:  |  Size: 6.8 KiB

BIN
src/models/ocr_model/train/dataset/images/11.png Normal file
View File

Binary file not shown.
Side by Side

After

Width:  |  Height:  |  Size: 4.1 KiB

BIN
src/models/ocr_model/train/dataset/images/12.png Normal file
View File

Binary file not shown.
Side by Side

After

Width:  |  Height:  |  Size: 5.2 KiB

BIN
src/models/ocr_model/train/dataset/images/13.png Normal file
View File

Binary file not shown.
Side by Side

After

Width:  |  Height:  |  Size: 12 KiB

BIN
src/models/ocr_model/train/dataset/images/14.png Normal file
View File

Binary file not shown.
Side by Side

After

Width:  |  Height:  |  Size: 2.8 KiB

BIN
src/models/ocr_model/train/dataset/images/15.png Normal file
View File

Binary file not shown.
Side by Side

After

Width:  |  Height:  |  Size: 2.2 KiB

BIN
src/models/ocr_model/train/dataset/images/16.png Normal file
View File

Binary file not shown.
Side by Side

After

Width:  |  Height:  |  Size: 2.2 KiB

BIN
src/models/ocr_model/train/dataset/images/17.png Normal file
View File

Binary file not shown.
Side by Side

After

Width:  |  Height:  |  Size: 2.6 KiB

BIN
src/models/ocr_model/train/dataset/images/18.png Normal file
View File

Binary file not shown.
Side by Side

After

Width:  |  Height:  |  Size: 3.1 KiB

BIN
src/models/ocr_model/train/dataset/images/19.png Normal file
View File

Binary file not shown.
Side by Side

After

Width:  |  Height:  |  Size: 2.7 KiB

BIN
src/models/ocr_model/train/dataset/images/2.png Normal file
View File

Binary file not shown.
Side by Side

After

Width:  |  Height:  |  Size: 3.9 KiB

BIN
src/models/ocr_model/train/dataset/images/20.png Normal file
View File

Binary file not shown.
Side by Side

After

Width:  |  Height:  |  Size: 3.9 KiB

BIN
src/models/ocr_model/train/dataset/images/21.png Normal file
View File

Binary file not shown.
Side by Side

After

Width:  |  Height:  |  Size: 2.9 KiB

BIN
src/models/ocr_model/train/dataset/images/22.png Normal file
View File

Binary file not shown.
Side by Side

After

Width:  |  Height:  |  Size: 3.7 KiB

BIN
src/models/ocr_model/train/dataset/images/23.png Normal file
View File

Binary file not shown.
Side by Side

After

Width:  |  Height:  |  Size: 3.5 KiB

BIN
src/models/ocr_model/train/dataset/images/24.png Normal file
View File

Binary file not shown.
Side by Side

After

Width:  |  Height:  |  Size: 3.1 KiB

BIN
src/models/ocr_model/train/dataset/images/25.png Normal file
View File

Binary file not shown.
Side by Side

After

Width:  |  Height:  |  Size: 2.5 KiB

BIN
src/models/ocr_model/train/dataset/images/26.png Normal file
View File

Binary file not shown.
Side by Side

After

Width:  |  Height:  |  Size: 2.2 KiB

BIN
src/models/ocr_model/train/dataset/images/27.png Normal file
View File

Binary file not shown.
Side by Side

After

Width:  |  Height:  |  Size: 3.1 KiB

BIN
src/models/ocr_model/train/dataset/images/28.png Normal file
View File

Binary file not shown.
Side by Side

After

Width:  |  Height:  |  Size: 2.9 KiB

BIN
src/models/ocr_model/train/dataset/images/29.png Normal file
View File

Binary file not shown.
Side by Side

After

Width:  |  Height:  |  Size: 5.3 KiB

BIN
src/models/ocr_model/train/dataset/images/3.png Normal file
View File

Binary file not shown.
Side by Side

After

Width:  |  Height:  |  Size: 4.1 KiB

BIN
src/models/ocr_model/train/dataset/images/30.png Normal file
View File

Binary file not shown.
Side by Side

After

Width:  |  Height:  |  Size: 3.9 KiB

BIN
src/models/ocr_model/train/dataset/images/31.png Normal file
View File

Binary file not shown.
Side by Side

After

Width:  |  Height:  |  Size: 4.9 KiB

BIN
src/models/ocr_model/train/dataset/images/32.png Normal file
View File

Binary file not shown.
Side by Side

After

Width:  |  Height:  |  Size: 2.9 KiB

BIN
src/models/ocr_model/train/dataset/images/33.png Normal file
View File

Binary file not shown.
Side by Side

After

Width:  |  Height:  |  Size: 1.8 KiB

BIN
src/models/ocr_model/train/dataset/images/34.png Normal file
View File

Binary file not shown.
Side by Side

After

Width:  |  Height:  |  Size: 3.2 KiB

BIN
src/models/ocr_model/train/dataset/images/4.png Normal file
View File

Binary file not shown.
Side by Side

After

Width:  |  Height:  |  Size: 5.7 KiB

BIN
src/models/ocr_model/train/dataset/images/5.png Normal file
View File

Binary file not shown.
Side by Side

After

Width:  |  Height:  |  Size: 11 KiB

BIN
src/models/ocr_model/train/dataset/images/6.png Normal file
View File

Binary file not shown.
Side by Side

After

Width:  |  Height:  |  Size: 4.8 KiB

BIN
src/models/ocr_model/train/dataset/images/7.png Normal file
View File

Binary file not shown.
Side by Side

After

Width:  |  Height:  |  Size: 4.5 KiB

BIN
src/models/ocr_model/train/dataset/images/8.png Normal file
View File

Binary file not shown.
Side by Side

After

Width:  |  Height:  |  Size: 2.5 KiB

BIN
src/models/ocr_model/train/dataset/images/9.png Normal file
View File

Binary file not shown.
Side by Side

After

Width:  |  Height:  |  Size: 5.2 KiB

50
src/models/ocr_model/train/dataset/loader.py Normal file
View File

@@ -0,0 +1,50 @@
from PIL import Image
from pathlib import Path
import datasets
import json
DIR_URL = Path('absolute/path/to/dataset/directory')
# e.g. DIR_URL = Path('/home/OleehyO/TeXTeller/src/models/ocr_model/train/dataset')
class LatexFormulas(datasets.GeneratorBasedBuilder):
BUILDER_CONFIGS = []
def _info(self):
return datasets.DatasetInfo(
features=datasets.Features({
"image": datasets.Image(),
"latex_formula": datasets.Value("string")
})
)
def _split_generators(self, dl_manager: datasets.DownloadManager):
dir_path = Path(dl_manager.download(str(DIR_URL)))
assert dir_path.is_dir()
return [
datasets.SplitGenerator(
name=datasets.Split.TRAIN,
gen_kwargs={
'dir_path': dir_path,
}
)
]
def _generate_examples(self, dir_path: Path):
images_path = dir_path / 'images'
formulas_path = dir_path / 'formulas.jsonl'
img2formula = {}
with formulas_path.open('r', encoding='utf-8') as f:
for line in f:
single_json = json.loads(line)
img2formula[single_json['img_name']] = single_json['formula']
for img_path in images_path.iterdir():
if img_path.suffix not in ['.jpg', '.png']:
continue
yield str(img_path), {
"image": Image.open(img_path),
"latex_formula": img2formula[img_path.name]
}

103
src/models/ocr_model/train/train.py Normal file
View File

@@ -0,0 +1,103 @@
import os
from functools import partial
from pathlib import Path
from datasets import load_dataset
from transformers import (
Trainer,
TrainingArguments,
Seq2SeqTrainer,
Seq2SeqTrainingArguments,
GenerationConfig
)
from .training_args import CONFIG
from ..model.TexTeller import TexTeller
from ..utils.functional import tokenize_fn, collate_fn, img_transform_fn
from ..utils.metrics import bleu_metric
from ...globals import MAX_TOKEN_SIZE, MIN_WIDTH, MIN_HEIGHT
def train(model, tokenizer, train_dataset, eval_dataset, collate_fn_with_tokenizer):
training_args = TrainingArguments(**CONFIG)
trainer = Trainer(
model,
training_args,
train_dataset=train_dataset,
eval_dataset=eval_dataset,
tokenizer=tokenizer,
data_collator=collate_fn_with_tokenizer,
)
trainer.train(resume_from_checkpoint=None)
def evaluate(model, tokenizer, eval_dataset, collate_fn):
eval_config = CONFIG.copy()
eval_config['predict_with_generate'] = True
generate_config = GenerationConfig(
max_new_tokens=MAX_TOKEN_SIZE,
num_beams=1,
do_sample=False,
pad_token_id=tokenizer.pad_token_id,
eos_token_id=tokenizer.eos_token_id,
bos_token_id=tokenizer.bos_token_id,
)
eval_config['generation_config'] = generate_config
seq2seq_config = Seq2SeqTrainingArguments(**eval_config)
trainer = Seq2SeqTrainer(
model,
seq2seq_config,
eval_dataset=eval_dataset,
tokenizer=tokenizer,
data_collator=collate_fn,
compute_metrics=partial(bleu_metric, tokenizer=tokenizer)
)
eval_res = trainer.evaluate()
print(eval_res)
if __name__ == '__main__':
script_dirpath = Path(__file__).resolve().parent
os.chdir(script_dirpath)
dataset = load_dataset(str(Path('./dataset/loader.py').resolve()))['train']
dataset = dataset.filter(lambda x: x['image'].height > MIN_HEIGHT and x['image'].width > MIN_WIDTH)
dataset = dataset.shuffle(seed=42)
dataset = dataset.flatten_indices()
tokenizer = TexTeller.get_tokenizer()
# If you want use your own tokenizer, please modify the path to your tokenizer
#+tokenizer = TexTeller.get_tokenizer('/path/to/your/tokenizer')
map_fn = partial(tokenize_fn, tokenizer=tokenizer)
tokenized_dataset = dataset.map(map_fn, batched=True, remove_columns=dataset.column_names, num_proc=8)
tokenized_dataset = tokenized_dataset.with_transform(img_transform_fn)
# Split dataset into train and eval, ratio 9:1
split_dataset = tokenized_dataset.train_test_split(test_size=0.1, seed=42)
train_dataset, eval_dataset = split_dataset['train'], split_dataset['test']
collate_fn_with_tokenizer = partial(collate_fn, tokenizer=tokenizer)
# Train from scratch
model = TexTeller()
# or train from TexTeller pre-trained model: model = TexTeller.from_pretrained()
# If you want to train from pre-trained model, please modify the path to your pre-trained checkpoint
#+e.g.
#+model = TexTeller.from_pretrained(
#+ '/path/to/your/model_checkpoint'
#+)
enable_train = True
enable_evaluate = False
if enable_train:
train(model, tokenizer, train_dataset, eval_dataset, collate_fn_with_tokenizer)
if enable_evaluate and len(eval_dataset) > 0:
evaluate(model, tokenizer, eval_dataset, collate_fn_with_tokenizer)

38
src/models/ocr_model/train/training_args.py Normal file
View File

@@ -0,0 +1,38 @@
CONFIG = {
"seed": 42, # Random seed for reproducibility
"use_cpu": False, # Whether to use CPU (it's easier to debug with CPU when starting to test the code)
"learning_rate": 5e-5, # Learning rate
"num_train_epochs": 10, # Total number of training epochs
"per_device_train_batch_size": 4, # Batch size per GPU for training
"per_device_eval_batch_size": 8, # Batch size per GPU for evaluation
"output_dir": "train_result", # Output directory
"overwrite_output_dir": False, # If the output directory exists, do not delete its content
"report_to": ["tensorboard"], # Report logs to TensorBoard
"save_strategy": "steps", # Strategy to save checkpoints
"save_steps": 500, # Interval of steps to save checkpoints, can be int or a float (0~1), when float it represents the ratio of total training steps (e.g., can set to 1.0 / 2000)
"save_total_limit": 5, # Maximum number of models to save. The oldest models will be deleted if this number is exceeded
"logging_strategy": "steps", # Log every certain number of steps
"logging_steps": 500, # Number of steps between each log
"logging_nan_inf_filter": False, # Record logs for loss=nan or inf
"optim": "adamw_torch", # Optimizer
"lr_scheduler_type": "cosine", # Learning rate scheduler
"warmup_ratio": 0.1, # Ratio of warmup steps in total training steps (e.g., for 1000 steps, the first 100 steps gradually increase lr from 0 to the set lr)
"max_grad_norm": 1.0, # For gradient clipping, ensure the norm of the gradients does not exceed 1.0 (default 1.0)
"fp16": False, # Whether to use 16-bit floating point for training (generally not recommended, as loss can easily explode)
"bf16": False, # Whether to use Brain Floating Point (bfloat16) for training (recommended if architecture supports it)
"gradient_accumulation_steps": 1, # Gradient accumulation steps, consider this parameter to achieve large batch size effects when batch size cannot be large
"jit_mode_eval": False, # Whether to use PyTorch jit trace during eval (can speed up the model, but the model must be static, otherwise will throw errors)
"torch_compile": False, # Whether to use torch.compile to compile the model (for better training and inference performance)
"dataloader_pin_memory": True, # Can speed up data transfer between CPU and GPU
"dataloader_num_workers": 1, # Default is not to use multiprocessing for data loading, usually set to 4*number of GPUs used
"evaluation_strategy": "steps", # Evaluation strategy, can be "steps" or "epoch"
"eval_steps": 500, # If evaluation_strategy="step"
"remove_unused_columns": False, # Don't change this unless you really know what you are doing.
}

46
src/models/ocr_model/utils/functional.py Normal file
View File

@@ -0,0 +1,46 @@
import torch
import numpy as np
from transformers import DataCollatorForLanguageModeling
from typing import List, Dict, Any
from .transforms import train_transform
def left_move(x: torch.Tensor, pad_val):
assert len(x.shape) == 2, 'x should be 2-dimensional'
lefted_x = torch.ones_like(x)
lefted_x[:, :-1] = x[:, 1:]
lefted_x[:, -1] = pad_val
return lefted_x
def tokenize_fn(samples: Dict[str, List[Any]], tokenizer=None) -> Dict[str, List[Any]]:
assert tokenizer is not None, 'tokenizer should not be None'
tokenized_formula = tokenizer(samples['latex_formula'], return_special_tokens_mask=True)
tokenized_formula['pixel_values'] = samples['image']
return tokenized_formula
def collate_fn(samples: List[Dict[str, Any]], tokenizer=None) -> Dict[str, List[Any]]:
assert tokenizer is not None, 'tokenizer should not be None'
pixel_values = [dic.pop('pixel_values') for dic in samples]
clm_collator = DataCollatorForLanguageModeling(tokenizer=tokenizer, mlm=False)
batch = clm_collator(samples)
batch['pixel_values'] = pixel_values
batch['decoder_input_ids'] = batch.pop('input_ids')
batch['decoder_attention_mask'] = batch.pop('attention_mask')
# left shift labels and decoder_attention_mask, padding with -100
batch['labels'] = left_move(batch['labels'], -100)
# convert list of Image to tensor with (B, C, H, W)
batch['pixel_values'] = torch.stack(batch['pixel_values'], dim=0)
return batch
def img_transform_fn(samples: Dict[str, List[Any]]) -> Dict[str, List[Any]]:
processed_img = train_transform(samples['pixel_values'])
samples['pixel_values'] = processed_img
return samples

26
src/models/ocr_model/utils/helpers.py Normal file
View File

@@ -0,0 +1,26 @@
import cv2
import numpy as np
from typing import List
def convert2rgb(image_paths: List[str]) -> List[np.ndarray]:
processed_images = []
for path in image_paths:
image = cv2.imread(path, cv2.IMREAD_UNCHANGED)
if image is None:
print(f"Image at {path} could not be read.")
continue
if image.dtype == np.uint16:
print(f'Converting {path} to 8-bit, image may be lossy.')
image = cv2.convertScaleAbs(image, alpha=(255.0/65535.0))
channels = 1 if len(image.shape) == 2 else image.shape[2]
if channels == 4:
image = cv2.cvtColor(image, cv2.COLOR_BGRA2RGB)
elif channels == 1:
image = cv2.cvtColor(image, cv2.COLOR_GRAY2RGB)
elif channels == 3:
image = cv2.cvtColor(image, cv2.COLOR_BGR2RGB)
processed_images.append(image)
return processed_images

43
src/models/ocr_model/utils/inference.py Normal file
View File

@@ -0,0 +1,43 @@
import torch
import numpy as np
from transformers import RobertaTokenizerFast, GenerationConfig
from typing import List, Union
from models.ocr_model.model.TexTeller import TexTeller
from models.ocr_model.utils.transforms import inference_transform
from models.ocr_model.utils.helpers import convert2rgb
from models.globals import MAX_TOKEN_SIZE
def inference(
model: TexTeller,
tokenizer: RobertaTokenizerFast,
imgs_path: Union[List[str], List[np.ndarray]],
use_cuda: bool,
num_beams: int = 1,
) -> List[str]:
model.eval()
if isinstance(imgs_path[0], str):
imgs = convert2rgb(imgs_path)
else: # already numpy array(rgb format)
assert isinstance(imgs_path[0], np.ndarray)
imgs = imgs_path
imgs = inference_transform(imgs)
pixel_values = torch.stack(imgs)
if use_cuda:
model = model.to('cuda')
pixel_values = pixel_values.to('cuda')
generate_config = GenerationConfig(
max_new_tokens=MAX_TOKEN_SIZE,
num_beams=num_beams,
do_sample=False,
pad_token_id=tokenizer.pad_token_id,
eos_token_id=tokenizer.eos_token_id,
bos_token_id=tokenizer.bos_token_id,
)
pred = model.generate(pixel_values, generation_config=generate_config)
res = tokenizer.batch_decode(pred, skip_special_tokens=True)
return res

23
src/models/ocr_model/utils/metrics.py Normal file
View File

@@ -0,0 +1,23 @@
import evaluate
import numpy as np
import os
from pathlib import Path
from typing import Dict
from transformers import EvalPrediction, RobertaTokenizer
def bleu_metric(eval_preds: EvalPrediction, tokenizer: RobertaTokenizer) -> Dict:
cur_dir = Path(os.getcwd())
os.chdir(Path(__file__).resolve().parent)
metric = evaluate.load('google_bleu') # Will download the metric from huggingface if not already downloaded
os.chdir(cur_dir)
logits, labels = eval_preds.predictions, eval_preds.label_ids
preds = logits
labels = np.where(labels == -100, 1, labels)
preds = tokenizer.batch_decode(preds, skip_special_tokens=True)
labels = tokenizer.batch_decode(labels, skip_special_tokens=True)
return metric.compute(predictions=preds, references=labels)

90
src/models/ocr_model/utils/transforms.py Normal file
View File

@@ -0,0 +1,90 @@
import torch
import random
import numpy as np
import cv2
from torchvision.transforms import v2
from typing import List
from PIL import Image
from models.globals import (
FIXED_IMG_SIZE,
IMAGE_MEAN, IMAGE_STD,
MAX_RESIZE_RATIO, MIN_RESIZE_RATIO
)
general_transform_pipeline = v2.Compose([
v2.ToImage(),
v2.ToDtype(torch.uint8, scale=True),
v2.Grayscale(),
v2.Resize(
size=FIXED_IMG_SIZE - 1,
interpolation=v2.InterpolationMode.BICUBIC,
max_size=FIXED_IMG_SIZE,
antialias=True
),
v2.ToDtype(torch.float32, scale=True),
v2.Normalize(mean=[IMAGE_MEAN], std=[IMAGE_STD]),
])
def trim_white_border(image: np.ndarray):
if len(image.shape) != 3 or image.shape[2] != 3:
raise ValueError("Image is not in RGB format or channel is not in third dimension")
if image.dtype != np.uint8:
raise ValueError(f"Image should stored in uint8")
h, w = image.shape[:2]
bg = np.full((h, w, 3), 255, dtype=np.uint8)
diff = cv2.absdiff(image, bg)
_, diff = cv2.threshold(diff, 1, 255, cv2.THRESH_BINARY)
gray_diff = cv2.cvtColor(diff, cv2.COLOR_RGB2GRAY)
x, y, w, h = cv2.boundingRect(gray_diff)
trimmed_image = image[y:y+h, x:x+w]
return trimmed_image
def padding(images: List[torch.Tensor], required_size: int):
images = [
v2.functional.pad(
img,
padding=[0, 0, required_size - img.shape[2], required_size - img.shape[1]]
)
for img in images
]
return images
def random_resize(
images: List[np.ndarray],
minr: float,
maxr: float
) -> List[np.ndarray]:
if len(images[0].shape) != 3 or images[0].shape[2] != 3:
raise ValueError("Image is not in RGB format or channel is not in third dimension")
ratios = [random.uniform(minr, maxr) for _ in range(len(images))]
return [
cv2.resize(img, (int(img.shape[1] * r), int(img.shape[0] * r)), interpolation=cv2.INTER_LANCZOS4) # 抗锯齿
for img, r in zip(images, ratios)
]
def general_transform(images: List[np.ndarray]) -> List[torch.Tensor]:
images = [trim_white_border(image) for image in images]
images = general_transform_pipeline(images)
images = padding(images, FIXED_IMG_SIZE)
return images
def train_transform(images: List[Image.Image]) -> List[torch.Tensor]:
images = [np.array(img.convert('RGB')) for img in images]
images = random_resize(images, MIN_RESIZE_RATIO, MAX_RESIZE_RATIO)
return general_transform(images)
def inference_transform(images: List[np.ndarray]) -> List[torch.Tensor]:
return general_transform(images)

25
src/models/tokenizer/train.py Normal file
View File

@@ -0,0 +1,25 @@
import os
from pathlib import Path
from datasets import load_dataset
from ..ocr_model.model.TexTeller import TexTeller
from ..globals import VOCAB_SIZE
if __name__ == '__main__':
script_dirpath = Path(__file__).resolve().parent
os.chdir(script_dirpath)
tokenizer = TexTeller.get_tokenizer()
# Don't forget to config your dataset path in loader.py
dataset = load_dataset('../ocr_model/train/dataset/loader.py')['train']
new_tokenizer = tokenizer.train_new_from_iterator(
text_iterator=dataset['latex_formula'],
# If you want to use a different vocab size, **change VOCAB_SIZE from globals.py**
vocab_size=VOCAB_SIZE
)
# Save the new tokenizer for later training and inference
new_tokenizer.save_pretrained('./your_dir_name')

87
src/server.py Normal file
View File

@@ -0,0 +1,87 @@
import argparse
import time
import numpy as np
import cv2
from starlette.requests import Request
from ray import serve
from ray.serve.handle import DeploymentHandle
from models.ocr_model.utils.inference import inference
from models.ocr_model.model.TexTeller import TexTeller
parser = argparse.ArgumentParser()
parser.add_argument(
'-ckpt', '--checkpoint_dir', type=str
)
parser.add_argument(
'-tknz', '--tokenizer_dir', type=str
)
parser.add_argument('-port', '--server_port', type=int, default=8000)
parser.add_argument('--num_replicas', type=int, default=1)
parser.add_argument('--ncpu_per_replica', type=float, default=1.0)
parser.add_argument('--ngpu_per_replica', type=float, default=0.0)
parser.add_argument('--use_cuda', action='store_true', default=False)
parser.add_argument('--num_beam', type=int, default=1)
args = parser.parse_args()
if args.ngpu_per_replica > 0 and not args.use_cuda:
raise ValueError("use_cuda must be True if ngpu_per_replica > 0")
@serve.deployment(
num_replicas=args.num_replicas,
ray_actor_options={
"num_cpus": args.ncpu_per_replica,
"num_gpus": args.ngpu_per_replica
}
)
class TexTellerServer:
def __init__(
self,
checkpoint_path: str,
tokenizer_path: str,
use_cuda: bool = False,
num_beam: int = 1
) -> None:
self.model = TexTeller.from_pretrained(checkpoint_path)
self.tokenizer = TexTeller.get_tokenizer(tokenizer_path)
self.use_cuda = use_cuda
self.num_beam = num_beam
self.model = self.model.to('cuda') if use_cuda else self.model
def predict(self, image_nparray) -> str:
return inference(self.model, self.tokenizer, [image_nparray], self.use_cuda, self.num_beam)[0]
@serve.deployment()
class Ingress:
def __init__(self, texteller_server: DeploymentHandle) -> None:
self.texteller_server = texteller_server
async def __call__(self, request: Request) -> str:
form = await request.form()
img_rb = await form['img'].read()
img_nparray = np.frombuffer(img_rb, np.uint8)
img_nparray = cv2.imdecode(img_nparray, cv2.IMREAD_COLOR)
img_nparray = cv2.cvtColor(img_nparray, cv2.COLOR_BGR2RGB)
pred = await self.texteller_server.predict.remote(img_nparray)
return pred
if __name__ == '__main__':
ckpt_dir = args.checkpoint_dir
tknz_dir = args.tokenizer_dir
serve.start(http_options={"port": args.server_port})
texteller_server = TexTellerServer.bind(ckpt_dir, tknz_dir, use_cuda=args.use_cuda, num_beam=args.num_beam)
ingress = Ingress.bind(texteller_server)
ingress_handle = serve.run(ingress, route_prefix="/predict")
while True:
time.sleep(1)

10
src/start_web.sh Executable file
View File

@@ -0,0 +1,10 @@
#!/usr/bin/env bash
set -exu
export CHECKPOINT_DIR="default"
export TOKENIZER_DIR="default"
# export USE_CUDA=False # True or False (case-sensitive)
export USE_CUDA=True # True or False (case-sensitive)
export NUM_BEAM=10
streamlit run web.py

231
src/web.py Normal file
View File

@@ -0,0 +1,231 @@
import os
import io
import base64
import tempfile
import time
import subprocess
import shutil
import streamlit as st
from PIL import Image, ImageChops
from pathlib import Path
from pdf2image import convert_from_path
from models.ocr_model.utils.inference import inference
from models.ocr_model.model.TexTeller import TexTeller
html_string = '''
<h1 style="color: black; text-align: center;">
<img src="https://slackmojis.com/emojis/429-troll/download" width="50">
TexTeller
<img src="https://slackmojis.com/emojis/429-troll/download" width="50">
</h1>
'''
suc_gif_html = '''
<h1 style="color: black; text-align: center;">
<img src="https://slackmojis.com/emojis/90621-clapclap-e/download" width="50">
<img src="https://slackmojis.com/emojis/90621-clapclap-e/download" width="50">
<img src="https://slackmojis.com/emojis/90621-clapclap-e/download" width="50">
</h1>
'''
fail_gif_html = '''
<h1 style="color: black; text-align: center;">
<img src="https://slackmojis.com/emojis/51439-allthethings_intensifies/download" >
<img src="https://slackmojis.com/emojis/51439-allthethings_intensifies/download" >
<img src="https://slackmojis.com/emojis/51439-allthethings_intensifies/download" >
</h1>
'''
tex = r'''
\documentclass{{article}}
\usepackage[
left=1in, % 左边距
right=1in, % 右边距
top=1in, % 上边距
bottom=1in,% 下边距
paperwidth=40cm, % 页面宽度
paperheight=40cm % 页面高度，这里以A4纸为例
]{{geometry}}
\usepackage[utf8]{{inputenc}}
\usepackage{{multirow,multicol,amsmath,amsfonts,amssymb,mathtools,bm,mathrsfs,wasysym,amsbsy,upgreek,mathalfa,stmaryrd,mathrsfs,dsfont,amsthm,amsmath,multirow}}
\begin{{document}}
{formula}
\pagenumbering{{gobble}}
\end{{document}}
'''
@st.cache_resource
def get_model():
return TexTeller.from_pretrained(os.environ['CHECKPOINT_DIR'])
@st.cache_resource
def get_tokenizer():
return TexTeller.get_tokenizer(os.environ['TOKENIZER_DIR'])
def get_image_base64(img_file):
buffered = io.BytesIO()
img_file.seek(0)
img = Image.open(img_file)
img.save(buffered, format="PNG")
return base64.b64encode(buffered.getvalue()).decode()
def rendering(formula: str, out_img_path: Path) -> bool:
build_dir = out_img_path / 'build'
build_dir.mkdir(exist_ok=True, parents=True)
f = build_dir / 'formula.tex'
f.touch(exist_ok=True)
f.write_text(tex.format(formula=formula))
p = subprocess.Popen([
'xelatex',
f'-output-directory={build_dir}',
'-interaction=nonstopmode',
'-halt-on-error',
f'{f}'
])
p.communicate()
return p.returncode == 0
def pdf_to_pngbytes(pdf_path):
images = convert_from_path(pdf_path, dpi=400,first_page=1, last_page=1)
trimmed_images = trim(images[0])
png_image_bytes = io.BytesIO()
trimmed_images.save(png_image_bytes, format='PNG')
png_image_bytes.seek(0)
return png_image_bytes
def trim(im):
bg = Image.new(im.mode, im.size, im.getpixel((0,0)))
diff = ImageChops.difference(im, bg)
diff = ImageChops.add(diff, diff, 2.0, -100)
bbox = diff.getbbox()
if bbox:
return im.crop(bbox)
return im
model = get_model()
tokenizer = get_tokenizer()
# check if xelatex is installed
xelatex_installed = os.system('which xelatex > /dev/null 2>&1') == 0
if "start" not in st.session_state:
st.session_state["start"] = 1
if xelatex_installed:
st.toast('Hooray!', icon='🎉')
time.sleep(0.5)
st.toast("Xelatex have been detected.", icon='✅')
else:
st.error('xelatex is not installed. Please install it before using TexTeller.')
# ============================ pages =============================== #
st.markdown(html_string, unsafe_allow_html=True)
uploaded_file = st.file_uploader("",type=['jpg', 'png', 'pdf'])
if xelatex_installed:
st.caption('🥳 Xelatex have been detected, rendered image will be displayed in the web page.')
else:
st.caption('😭 Xelatex is not detected, please check the resulting latex code by yourself, or check ... to have your xelatex setup ready.')
if uploaded_file:
img = Image.open(uploaded_file)
temp_dir = tempfile.mkdtemp()
png_file_path = os.path.join(temp_dir, 'image.png')
img.save(png_file_path, 'PNG')
img_base64 = get_image_base64(uploaded_file)
st.markdown(f"""
<style>
.centered-container {{
text-align: center;
}}
.centered-image {{
display: block;
margin-left: auto;
margin-right: auto;
max-width: 500px;
max-height: 500px;
}}
</style>
<div class="centered-container">
<img src="data:image/png;base64,{img_base64}" class="centered-image" alt="Input image">
<p style="color:gray;">Input image ({img.height}✖️{img.width})</p>
</div>
""", unsafe_allow_html=True)
st.write("")
with st.spinner("Predicting..."):
uploaded_file.seek(0)
TeXTeller_result = inference(
model,
tokenizer,
[png_file_path],
True if os.environ['USE_CUDA'] == 'True' else False,
int(os.environ['NUM_BEAM'])
)[0]
if not xelatex_installed:
st.markdown(fail_gif_html, unsafe_allow_html=True)
st.warning('Unable to find xelatex to render image. Please check the prediction results yourself.', icon="🤡")
txt = st.text_area(
":red[Predicted formula]",
TeXTeller_result,
height=150,
)
else:
is_successed = rendering(TeXTeller_result, Path(temp_dir))
if is_successed:
# st.code(TeXTeller_result, language='latex')
img_base64 = get_image_base64(pdf_to_pngbytes(Path(temp_dir) / 'build' / 'formula.pdf'))
st.markdown(suc_gif_html, unsafe_allow_html=True)
st.success('Successfully rendered!', icon="✅")
txt = st.text_area(
":red[Predicted formula]",
TeXTeller_result,
height=150,
)
# st.latex(TeXTeller_result)
st.markdown(f"""
<style>
.centered-container {{
text-align: center;
}}
.centered-image {{
display: block;
margin-left: auto;
margin-right: auto;
max-width: 500px;
max-height: 500px;
}}
</style>
<div class="centered-container">
<img src="data:image/png;base64,{img_base64}" class="centered-image" alt="Input image">
</div>
""", unsafe_allow_html=True)
else:
st.markdown(fail_gif_html, unsafe_allow_html=True)
st.error('Rendering failed. You can try using a higher resolution image or splitting the multi line formula into a single line for better results.', icon="❌")
txt = st.text_area(
":red[Predicted formula]",
TeXTeller_result,
height=150,
)
shutil.rmtree(temp_dir)
# ============================ pages =============================== #