TexTellerv2 release
This commit is contained in:
三洋三洋
16
README.md
16
README.md
@@ -1,3 +1,5 @@
|
||||
📄 English | <a href="./assets/README_zh.md">中文</a>
|
||||
|
||||
<div align="center">
|
||||
<h1>
|
||||
<img src="./assets/fire.svg" width=30, height=30>
|
||||
@@ -5,7 +7,7 @@
|
||||
<img src="./assets/fire.svg" width=30, height=30>
|
||||
</h1>
|
||||
<p align="center">
|
||||
English | <a href="./assets/README_zh.md">中文</a>
|
||||
🤗 <a href="https://huggingface.co/OleehyO/TexTeller"> Hugging Face</a>
|
||||
</p>
|
||||
<!-- <p align="center">
|
||||
<img src="./assets/web_demo.gif" alt="TexTeller_demo" width=800>
|
||||
@@ -22,14 +24,14 @@ TexTeller was trained with ~~550K~~7.5M image-formula pairs (dataset available [
|
||||
|
||||
## 🔄 Change Log
|
||||
|
||||
* 📮[2024-03-24] TexTeller 2.0 released! The training data for TexTeller 2.0 has been increased to 7.5M (about **15 times more** than TexTeller 1.0 and also improved in data quality). The trained TexTeller 2.0 demonstrated **superior performance** in the test set, especially in recognizing rare symbols, complex multi-line formulas, and matrices.
|
||||
* 📮[2024-03-25] TexTeller 2.0 released! The training data for TexTeller 2.0 has been increased to 7.5M (about **15 times more** than TexTeller 1.0 and also improved in data quality). The trained TexTeller 2.0 demonstrated **superior performance** in the test set, especially in recognizing rare symbols, complex multi-line formulas, and matrices.
|
||||
> [There](./assets/test.pdf) are more test images here and a horizontal comparison of recognition models from different companies.
|
||||
|
||||
## 🔑 Prerequisites
|
||||
|
||||
python=3.10
|
||||
|
||||
pytorch
|
||||
[pytorch](https://pytorch.org/get-started/locally/)
|
||||
|
||||
> [!WARNING]
|
||||
> Only CUDA versions >= 12.0 have been fully tested, so it is recommended to use CUDA version >= 12.0
|
||||
@@ -71,19 +73,21 @@ pytorch
|
||||
|
||||
## 🌐 Web Demo
|
||||
|
||||
To start the web demo, you need to first enter the `TexTeller/src` directory, then run the following command
|
||||
First, **ensure that [poppler](https://poppler.freedesktop.org/) is correctly installed and added to the `PATH`** (so that the `pdftoppm` command can be directly used in the terminal).
|
||||
|
||||
Then, go to the `TexTeller/src` directory and run the following command:
|
||||
|
||||
```bash
|
||||
./start_web.sh
|
||||
```
|
||||
|
||||
Then, enter `http://localhost:8501` in your browser to see the web demo
|
||||
Enter `http://localhost:8501` in a browser to view the web demo.
|
||||
|
||||
> [!TIP]
|
||||
> You can change the default configuration of `start_web.sh`, for example, to use GPU for inference (e.g. `USE_CUDA=True`) or to increase the number of beams (e.g. `NUM_BEAM=3`) to achieve higher accuracy
|
||||
|
||||
> [!IMPORTANT]
|
||||
> If you want to directly render the prediction results as images on the web (for example, to check if the prediction is correct), you need to ensure [xelatex is correctly installed](https://github.com/OleehyO/TexTeller?tab=readme-ov-file#Rendering-Predicted-Results)
|
||||
> If you want to directly render the prediction results as images on the web (for example, to check if the prediction is correct), you need to ensure [xelatex is correctly installed](https://github.com/OleehyO/TexTeller?tab=readme-ov-file#-about-rendering-latex-as-images)
|
||||
|
||||
## 📡 API Usage
|
||||
|
||||
|
||||
@@ -1,3 +1,5 @@
|
||||
📄 <a href="../README.md">English</a> | 中文
|
||||
|
||||
<div align="center">
|
||||
<h1>
|
||||
<img src="./fire.svg" width=30, height=30>
|
||||
@@ -5,7 +7,7 @@
|
||||
<img src="./fire.svg" width=30, height=30>
|
||||
</h1>
|
||||
<p align="center">
|
||||
<a href="../README.md">English</a> | 中文
|
||||
🤗 <a href="https://huggingface.co/OleehyO/TexTeller">Hugging Face</a>
|
||||
</p>
|
||||
<!-- <p align="center">
|
||||
<img src="./web_demo.gif" alt="TexTeller_demo" width=800>
|
||||
@@ -22,14 +24,14 @@ TexTeller用了~~550K~~7.5M的图片-公式对进行训练(数据集可以在[
|
||||
|
||||
## 🔄 变更信息
|
||||
|
||||
* 📮[2024-03-24] TexTeller2.0发布！TexTeller2.0的训练数据增大到了7.5M(相较于TexTeller1.0**增加了~15倍**并且数据质量也有所改善)。训练后的TexTeller2.0在测试集中展现出了**更加优越的性能**，尤其在生僻符号、复杂多行、矩阵的识别场景中。
|
||||
* 📮[2024-03-25] TexTeller2.0发布！TexTeller2.0的训练数据增大到了7.5M(相较于TexTeller1.0**增加了~15倍**并且数据质量也有所改善)。训练后的TexTeller2.0在测试集中展现出了**更加优越的性能**，尤其在生僻符号、复杂多行、矩阵的识别场景中。
|
||||
> 在[这里](./test.pdf)有更多的测试图片以及各家识别模型的横向对比。
|
||||
|
||||
## 🔑 前置条件
|
||||
|
||||
python=3.10
|
||||
|
||||
pytorch
|
||||
[pytorch](https://pytorch.org/get-started/locally/)
|
||||
|
||||
> [!WARNING]
|
||||
> 只有CUDA版本>= 12.0被完全测试过，所以最好使用>= 12.0的CUDA版本
|
||||
@@ -99,19 +101,21 @@ pytorch
|
||||
|
||||
## 🌐 网页演示
|
||||
|
||||
要想启动web demo，你需要先进入 `TexTeller/src` 目录，然后运行以下命令
|
||||
首先**确保[poppler](https://poppler.freedesktop.org/)被正确安装，并添加到`PATH`路径中**（终端可以直接使用`pdftoppm`命令）。
|
||||
|
||||
然后进入 `TexTeller/src` 目录，运行以下命令
|
||||
|
||||
```bash
|
||||
./start_web.sh
|
||||
```
|
||||
|
||||
然后在浏览器里输入`http://localhost:8501`就可以看到web demo
|
||||
在浏览器里输入`http://localhost:8501`就可以看到web demo
|
||||
|
||||
> [!TIP]
|
||||
> 你可以改变`start_web.sh`的默认配置， 例如使用GPU进行推理(e.g. `USE_CUDA=True`) 或者增加beams的数量(e.g. `NUM_BEAM=3`)来获得更高的精确度
|
||||
|
||||
> [!IMPORTANT]
|
||||
> 如果你想直接把预测结果在网页上渲染成图片（比如为了检查预测结果是否正确）你需要确保[xelatex被正确安装](https://github.com/OleehyO/TexTeller?tab=readme-ov-file#Rendering-Predicted-Results)
|
||||
> 如果你想直接把预测结果在网页上渲染成图片（比如为了检查预测结果是否正确）你需要确保[xelatex被正确安装](https://github.com/OleehyO/TexTeller?tab=readme-ov-file#-关于把latex渲染成图片)
|
||||
|
||||
## 📡 API调用
|
||||
|
||||
@@ -150,8 +154,7 @@ python server.py # default settings
|
||||
如果你使用了不一样的数据集，你可能需要重新训练tokenizer来得到一个不一样的字典。配置好数据集后，可以通过以下命令来训练自己的tokenizer：
|
||||
|
||||
1. 在`TexTeller/src/models/tokenizer/train.py`中，修改`new_tokenizer.save_pretrained('./your_dir_name')`为你自定义的输出目录
|
||||
> [!IMPORTANT]
|
||||
> 如果要用一个不一样大小的字典(默认1W个token)，你需要在 `TexTeller/src/models/globals.py`中修改`VOCAB_SIZE`变量
|
||||
> 注意：如果要用一个不一样大小的字典(默认1W个token)，你需要在 `TexTeller/src/models/globals.py`中修改`VOCAB_SIZE`变量
|
||||
|
||||
2. **在 `TexTeller/src` 目录下**运行以下命令:
|
||||
|
||||
|
||||
@@ -17,7 +17,7 @@ from transformers import (
|
||||
|
||||
|
||||
class TexTeller(VisionEncoderDecoderModel):
|
||||
REPO_NAME = '/home/lhy/code/TexTeller/src/models/ocr_model/train/train_result/TexTellerv2/checkpoint-588000'
|
||||
REPO_NAME = 'OleehyO/TexTeller'
|
||||
def __init__(self, decoder_path=None, tokenizer_path=None):
|
||||
encoder = ViTModel(ViTConfig(
|
||||
image_size=FIXED_IMG_SIZE,
|
||||
|
||||
@@ -3,8 +3,7 @@ set -exu
|
||||
|
||||
export CHECKPOINT_DIR="default"
|
||||
export TOKENIZER_DIR="default"
|
||||
# export USE_CUDA=False # True or False (case-sensitive)
|
||||
export USE_CUDA=True # True or False (case-sensitive)
|
||||
export NUM_BEAM=10
|
||||
export USE_CUDA=False # True or False (case-sensitive)
|
||||
export NUM_BEAM=1
|
||||
|
||||
streamlit run web.py
|
||||
|
||||
@@ -65,7 +65,6 @@ tex = r'''
|
||||
def get_model():
|
||||
return TexTeller.from_pretrained(os.environ['CHECKPOINT_DIR'])
|
||||
|
||||
|
||||
@st.cache_resource
|
||||
def get_tokenizer():
|
||||
return TexTeller.get_tokenizer(os.environ['TOKENIZER_DIR'])
|
||||
|
||||
Reference in New Issue
Block a user