TexTellerv2 release

README.md

@@ -1,3 +1,5 @@
+📄 English | <a href="./assets/README_zh.md">中文</a>
+
 <div align="center">
     <h1>
         <img src="./assets/fire.svg" width=30, height=30> 
@@ -5,7 +7,7 @@
         <img src="./assets/fire.svg" width=30, height=30>
     </h1>
     <p align="center">
-        English | <a href="./assets/README_zh.md">中文</a>
+        🤗 <a href="https://huggingface.co/OleehyO/TexTeller"> Hugging Face</a>
     </p>
     <!-- <p align="center">
         <img src="./assets/web_demo.gif" alt="TexTeller_demo" width=800>
@@ -22,14 +24,14 @@ TexTeller was trained with ~~550K~~7.5M image-formula pairs (dataset available [
 
 ## 🔄 Change Log
 
-* 📮[2024-03-24] TexTeller 2.0 released! The training data for TexTeller 2.0 has been increased to 7.5M (about **15 times more** than TexTeller 1.0 and also improved in data quality). The trained TexTeller 2.0 demonstrated **superior performance** in the test set, especially in recognizing rare symbols, complex multi-line formulas, and matrices.
+* 📮[2024-03-25] TexTeller 2.0 released! The training data for TexTeller 2.0 has been increased to 7.5M (about **15 times more** than TexTeller 1.0 and also improved in data quality). The trained TexTeller 2.0 demonstrated **superior performance** in the test set, especially in recognizing rare symbols, complex multi-line formulas, and matrices.
     > [There](./assets/test.pdf) are more test images here and a horizontal comparison of recognition models from different companies.
 
 ## 🔑 Prerequisites
 
 python=3.10
 
-pytorch
+[pytorch](https://pytorch.org/get-started/locally/)
 
 > [!WARNING]
 > Only CUDA versions >= 12.0 have been fully tested, so it is recommended to use CUDA version >= 12.0
@@ -71,19 +73,21 @@ pytorch
 
 ## 🌐 Web Demo
 
-To start the web demo, you need to first enter the `TexTeller/src` directory, then run the following command
+First, **ensure that [poppler](https://poppler.freedesktop.org/) is correctly installed and added to the `PATH`** (so that the `pdftoppm` command can be directly used in the terminal).
+
+Then, go to the `TexTeller/src` directory and run the following command:
 
 ```bash
 ./start_web.sh
 ```
 
-Then, enter `http://localhost:8501` in your browser to see the web demo
+Enter `http://localhost:8501` in a browser to view the web demo.
 
 > [!TIP]
 > You can change the default configuration of `start_web.sh`, for example, to use GPU for inference (e.g. `USE_CUDA=True`) or to increase the number of beams (e.g. `NUM_BEAM=3`) to achieve higher accuracy
 
 > [!IMPORTANT]
-> If you want to directly render the prediction results as images on the web (for example, to check if the prediction is correct), you need to ensure [xelatex is correctly installed](https://github.com/OleehyO/TexTeller?tab=readme-ov-file#Rendering-Predicted-Results)
+> If you want to directly render the prediction results as images on the web (for example, to check if the prediction is correct), you need to ensure [xelatex is correctly installed](https://github.com/OleehyO/TexTeller?tab=readme-ov-file#-about-rendering-latex-as-images)
 
 ## 📡 API Usage
 

assets/README_zh.md

@@ -1,3 +1,5 @@
+📄 <a href="../README.md">English</a> | 中文
+
 <div align="center">
     <h1>
         <img src="./fire.svg" width=30, height=30> 
@@ -5,7 +7,7 @@
         <img src="./fire.svg" width=30, height=30> 
     </h1>
     <p align="center">
-        <a href="../README.md">English</a> | 中文
+        🤗 <a href="https://huggingface.co/OleehyO/TexTeller">Hugging Face</a>
     </p>
     <!-- <p align="center">
         <img src="./web_demo.gif" alt="TexTeller_demo" width=800>
@@ -22,14 +24,14 @@ TexTeller用了~~550K~~7.5M的图片-公式对进行训练(数据集可以在[
 
 ## 🔄 变更信息
 
-* 📮[2024-03-24] TexTeller2.0发布！TexTeller2.0的训练数据增大到了7.5M(相较于TexTeller1.0**增加了~15倍**并且数据质量也有所改善)。训练后的TexTeller2.0在测试集中展现出了**更加优越的性能**，尤其在生僻符号、复杂多行、矩阵的识别场景中。
+* 📮[2024-03-25] TexTeller2.0发布！TexTeller2.0的训练数据增大到了7.5M(相较于TexTeller1.0**增加了~15倍**并且数据质量也有所改善)。训练后的TexTeller2.0在测试集中展现出了**更加优越的性能**，尤其在生僻符号、复杂多行、矩阵的识别场景中。
     > 在[这里](./test.pdf)有更多的测试图片以及各家识别模型的横向对比。
 
 ## 🔑 前置条件
 
 python=3.10
 
-pytorch
+[pytorch](https://pytorch.org/get-started/locally/)
 
 > [!WARNING]
 > 只有CUDA版本>= 12.0被完全测试过，所以最好使用>= 12.0的CUDA版本
@@ -99,19 +101,21 @@ pytorch
 
 ## 🌐 网页演示
 
-要想启动web demo，你需要先进入 `TexTeller/src` 目录，然后运行以下命令
+首先**确保[poppler](https://poppler.freedesktop.org/)被正确安装，并添加到`PATH`路径中**（终端可以直接使用`pdftoppm`命令）。
+
+然后进入 `TexTeller/src` 目录，运行以下命令
 
 ```bash
 ./start_web.sh
 ```
 
-然后在浏览器里输入`http://localhost:8501`就可以看到web demo
+在浏览器里输入`http://localhost:8501`就可以看到web demo
 
 > [!TIP]
 > 你可以改变`start_web.sh`的默认配置， 例如使用GPU进行推理(e.g. `USE_CUDA=True`) 或者增加beams的数量(e.g. `NUM_BEAM=3`)来获得更高的精确度
 
 > [!IMPORTANT]
-> 如果你想直接把预测结果在网页上渲染成图片（比如为了检查预测结果是否正确）你需要确保[xelatex被正确安装](https://github.com/OleehyO/TexTeller?tab=readme-ov-file#Rendering-Predicted-Results)
+> 如果你想直接把预测结果在网页上渲染成图片（比如为了检查预测结果是否正确）你需要确保[xelatex被正确安装](https://github.com/OleehyO/TexTeller?tab=readme-ov-file#-关于把latex渲染成图片)
 
 ## 📡 API调用
 
@@ -150,8 +154,7 @@ python server.py  # default settings
 如果你使用了不一样的数据集，你可能需要重新训练tokenizer来得到一个不一样的字典。配置好数据集后，可以通过以下命令来训练自己的tokenizer：
 
 1. 在`TexTeller/src/models/tokenizer/train.py`中，修改`new_tokenizer.save_pretrained('./your_dir_name')`为你自定义的输出目录
-    > [!IMPORTANT]
+    > 注意：如果要用一个不一样大小的字典(默认1W个token)，你需要在 `TexTeller/src/models/globals.py`中修改`VOCAB_SIZE`变量
-    > 如果要用一个不一样大小的字典(默认1W个token)，你需要在 `TexTeller/src/models/globals.py`中修改`VOCAB_SIZE`变量
 
 2. **在 `TexTeller/src` 目录下**运行以下命令:
 

src/models/ocr_model/model/TexTeller.py

@@ -17,7 +17,7 @@ from transformers import (
 
 
 class TexTeller(VisionEncoderDecoderModel):
-    REPO_NAME = '/home/lhy/code/TexTeller/src/models/ocr_model/train/train_result/TexTellerv2/checkpoint-588000'
+    REPO_NAME = 'OleehyO/TexTeller'
     def __init__(self, decoder_path=None, tokenizer_path=None):
         encoder = ViTModel(ViTConfig(
             image_size=FIXED_IMG_SIZE,

src/start_web.sh

@@ -3,8 +3,7 @@ set -exu
 
 export CHECKPOINT_DIR="default"
 export TOKENIZER_DIR="default"
-# export USE_CUDA=False  # True or False (case-sensitive)
+export USE_CUDA=False  # True or False (case-sensitive)
-export USE_CUDA=True  # True or False (case-sensitive)
+export NUM_BEAM=1
-export NUM_BEAM=10
 
 streamlit run web.py

src/web.py

@@ -65,7 +65,6 @@ tex = r'''
 def get_model():
     return TexTeller.from_pretrained(os.environ['CHECKPOINT_DIR'])
 
-
 @st.cache_resource
 def get_tokenizer():
     return TexTeller.get_tokenizer(os.environ['TOKENIZER_DIR'])