• This is a modified branch of project: Franklin-Zhang0/Yolo-v8-Apex-Aim-assist. Kudos to Franklin! If you find this project useful, please consider starring the original repo!
• This project has no intend of being utilized in game play and it should not be. Use at your own risk.
• This project will not be regularly maintained as it already meets the goal(for now). Update might be provided once better models/compression methods/quantization methods are available.

- OS: Windows 11 Pro 22H2
- CPU: AMD Ryzen 3900X
- GPU: NVIDIA GeForce RTX 2080 8G
- RAM: 32G @ 3600MHz
- Monitor resolution: 2560 x 1440
- In-game resolution: 2560 x 1440
- In-game FPS: Locked @ 60fps

Advantages:

• Fast screencapturing with dxshot
• Not Logitech Macro/GHub dependent
• Customizable trigger keys
• PID capable
• IFF(Identification friend or foe) capable (Works 50/50)
• fp16 precision

Todos:

• int8 precision (No significant improvement as for now)
• Pruning (Sacrificies accuracy way too much)
• Increase accuracy of IFF
• Increase accuracy under: ADS, partial body exposure, gunfire blockage, smoke...

• Install Conda (if not already installed)

  In your terminal window, run:bash Miniconda3-latest-Linux-x86_64.sh to install Miniconda (suggested) or run:bash Anaconda-latest-Linux-x86_64.sh to install Anaconda.

• Setup a new environment

  In your terminal window, run:

  conda create -n yolov8 python=3.10 # create environment 'yolov8' with python 3.10 installed 
  conda activate yolov8 # activate environment 'yolov8'

• Install CUDA and PyTorch.

  if you have a cuda capable gpu, you can running the following extra command

  Running inferrence on CPU is suboptimal and no longer suggested by this project. Please use CUDA capable GPU and run the following command to install CUDA, cuDNN and PyTorch:

  pip install torch==1.13.1+cu116 torchvision==0.14.1+cu116 torchaudio==0.13.1 --extra-index-url https://download.pytorch.org/whl/cu116`

• Install TensorRT.

  Add the following command

  pip install --upgrade setuptools pip --user
  pip install nvidia-pyindex
  pip install --upgrade nvidia-tensorrt
  pip install pycuda
  

• Install python requirement.

  pip install -r requirement.txt

The following method has being tested and successed under Windows 10 Pro Version 21H2/22H2, Windows11 Pro Version 22H2 and Windows11 Pro Insider View Build 25346. Technically, it works under all latest Windows OS builds.

• Version-align

  CUDA	cuDNN	TensorRT	PyTorch
  11.7.0	8.5.0	8.5.2.2	2.0.0
  11.8.0	8.6.0	8.5.3.1	2.0.0
  ...	...	...	...

  We will be using the first row as our package manifests.

• Install CUDA. (One can also follow the official instruction:CUDA official website).

  conda install cuda -c nvidia/label/cuda-11.7.0 # install CUDA 11.7.0

• Install cuDNN.

  • Register for the NVIDIA developer program.
  • Go to the cuDNN download site:cuDNN download archive.
  • Click Download cuDNN v8.5.0 (August 8th, 2022), for CUDA 11.x.
  • Download Local Installer for Windows (Zip).
  • Unzip cudnn-windows-x86_64-8.5.0.96_cuda11-archive.zip.
  • Copy all three folders (bin,include,lib) and paste them to the CUDA installation directory C:\Program Files\NVIDIA GPU Computing Toolkit\CUDA\v11.7. (NOTE bin,include,lib folders are already exist in the CUDA folder.).

• Install PyTorch.

  conda install pytorch torchvision torchaudio pytorch-cuda=11.7 -c pytorch -c nvidia

• Install TensorRT. Follow the Nvidia instruction of installation.

  • Go to the TensorRT download site.
  • Download TensorRT 8.5 GA Update 1 for x86_64 Architecture for Windows and CUDA 11.0, 11.1, 11.2, 11.3, 11.4, 11.5, 11.6, 11.7 and 11.8 ZIP Package.
  • Unzip the folder TensorRT-8.5.2.2 from TensorRT-8.5.2.2.Windows10.x86_64.cuda-11.8.cudnn8.6.zip.
  • Add the <your install path>\TensorRT-8.5.2.2\lib into PATH environment variable.
  • Go to the folder <your install path>\TensorRT-8.5.2.2\python
  • In command window, input

    conda activate yolov8 # activate dedicated environment
    pip install tensorrt-8.5.2.2-cp310-none-win_amd64.whl # install tensorrt package to python

• Install python requirement.

  pip install -r requirements.txt

You have several options here to realize your .pt weight:

• (1) Use the provided weight apex_v8n.pt.

  This is a weight based on yolov8n.pt, trained with 7W screenshots and labeled 'Enemy', 'Teammate'. Due to the nature of this project, the provided weight is poorly trained to prevent abuse and cheating. However, it can already track characters well and demonstrate some level of IFF capability.

• (2) Use the provided weight apex_v8n.pt as a pretrained weight and train your own weight with your own dataset.

  Please follow the official instruction of Ultralytics to train your own weight.

  Note that the dataset is required to use YOLO annotation format, please reform your dataset into the following structure:

  dataset/apex/
  ├── train
  |   ├── images
  │   |   ├── 000000000001.jpg
  │   |   ├── 000000580008.jpg
  │   |   |   ...
  |   ├── lables    
  │   |   ├── 000000000001.txt
  │   |   ├── 000000580008.txt
  │   |   |   ...
  ├── valid
  |   ├── images
  │   |   ├── 000000000113.jpg
  │   |   ├── 000000000567.jpg
  |   ├── lables    
  │   |   ├── 000000000113.txt
  │   |   ├── 000000000567.txt
  │   |   |   ...    

• (3) Train your own weight with official pretrained yolov8 weights.

  If the provided weight which is based on yolov8n.pt can not meet your expectation. You can also explore the options of other pretrained weights provided by yolov8.

  • Model speed: 8n>8s>8m

  • Model accuracy: 8n<8s<8m

  Please follow the official instruction of Ultralytics to train your own weight.

You have several options here to to convert your .pt weight to a .onnx weight.

• (1) Use yolov8 built in function YOLO export:

  yolo export model=<your weight path>/best.pt format=onnx

  Note this built-in method is identical to the python code provided in TensorRT-For-YOLO-Series

• (2) Use Paddleslim ACT (In Linux):

Use any of the following methods to generate TensorRT engine:

• (1) Use yolov8 built-in function YOLO export to export .engine weight directly from .pt weight.

  # out put fp32 precision (default)
  yolo export model=<your weight path>/best.pt format=engine
  
  # out put fp16 precision (recommanded)
  yolo export model=<your weight path>/best.pt format=engine fp16=True

• (2) Use the third-party method TensorRT-For-YOLO-Series to export .trt weight from previous generated .onnx weight.

  # out put fp32 precision
  python export.py -o <your weight path>/best.onnx -e apex_fp32.trt -p fp32 --end2end --v8
  
  # out put fp16 precision (default, recommanded)
  python export.py -o <your weight path>/best.onnx -e apex_fp16.trt --end2end --v8
  
  # out put int8 precision (for extreme performance)
  python export.py -o <your weight path>/best.onnx -e apex_int8.trt -p int8 --calib_input <your data path>/train/images --calib_num_images 500 --end2end --v8

Replace the model and change settings in the args_.py file (See Section 4.)

Run the main.py file with the following command

conda activate v8trt # activate dedeicated env
python main.py # start aim assist

After a few seconds, the program will start fucntioning. You should see the follwing prompts in the console:

listener start
Main start
Screenshot fps:  311.25682541048934
fps:  61.09998811302416
interval:  0.016366615295410156

Explaination of keys:

• Shift: Holding shift triggers aim assist. By default, only holding shift can trigger the aim assist.
• Left: Unlock left mouse button. Once left key is clicked, you should hear a beep and now holding left mouse button can also trigger aim assist
• Right: Unlock right mouse button. Once right key is clicked, you should hear a beep and now holding right mouse button can also trigger aim assist
• End: Cilck End for continious aiming. Auto aiming is always on and another click to turn off.
• Left mouse button: Optional trigger
• Right mouse button: Optional trigger
• Home: Stop listening and exit program

You can change following settings in the args_.py file.

• --model: The weight to be used by this project. Please replace this with your own .trt or .engine weight.
• --classes: classes to be detected, can be expanded but need to be an array. For example, 0 represents 'Teammate', 1 represents 'Enemy'. Then the input should be [1].
• --conf: Confidence level for inference. Adjust it based on your model accuracy.
• --crop_size: The portion to detect from the screen. Adjust it based on your monitor resolution. For example: 1/3 for 1440P, or 1/2 for 1080P.
• --pid: Use PID controller to smooth aiming and prevent overdrifting. Leave it by default is recommanded.
• --Kp,--Ki,--Kd: PID components that need to be carefully calibrated. --Kp=0.5, --Ki=0.1, --Kd=0.1 is recommanded as starting point. You can also modify the MyListener.py file.
• Function def listen_key(key) and def keys_released(key): Change keyboard.Key.home, keyboard.Key.end, keyboard.Key.shift, keyboard.Key.left or keyboard.Key.right to whatever keys you like to customize the key settings.

Train image dataset

TensorRt code