Skip to content

CLI to tidy ({renumber,move-to-bottom,sort,clean}) up {image,link} references for markdown

License

Notifications You must be signed in to change notification settings

realazthat/mdreftidy

Repository files navigation

mdreftidy

Audience: Developers Platform: Linux

🎇Features  •  🏠Installation  •  🚜Usage  •  💻CLI  •  💡Examples

✅Requirements  •  🐳Docker

Top language GitHub License PyPI - Version Python Version

CLI to tidy ({renumber,move-to-bottom,sort,clean}) up {image,link} references for markdown


Status Stable Unstable
Master Build and Test since tagged last commit
Develop Build and Test since tagged since tagged last commit

Demo

❔ What

What mdreftidy does:

Turn this (./mdreftidy/examples/SIMPLE.md):

# Example markdown file

## Reference link: Out of order

[Reference link: Out of order 1][3], [Reference link: Out of order 2][2].

[2]: ./reference-link-out-of-order-2

## Footnotes

[6]: ./unused-reference-link-6
[3]: ./reference-link-out-of-order-3
[1]: ./unused-reference-link-1

Into this (./mdreftidy/examples/SIMPLE.tidied.md):

# Example markdown file

## Reference link: Out of order

[Reference link: Out of order 1][1], [Reference link: Out of order 2][2].


## Footnotes

[1]: ./reference-link-out-of-order-3
[2]: ./reference-link-out-of-order-2

🎇 Features

  • Renumbers references by order used.
  • Optionally removes unused references.
  • Optionally moves references to the bottom.
  • Optionally sorts reference blocks.

🏠 Installation

# Install from pypi (https://pypi.org/project/mdreftidy/)
pip install mdreftidy

# Install from git (https://github.com/realazthat/mdreftidy)
pip install git+https://github.com/realazthat/[email protected]

🚜 Usage

Example README: (./mdreftidy/examples/SIMPLE.md):

# Example markdown file

## Reference link: Out of order

[Reference link: Out of order 1][3], [Reference link: Out of order 2][2].

[2]: ./reference-link-out-of-order-2

## Footnotes

[6]: ./unused-reference-link-6
[3]: ./reference-link-out-of-order-3
[1]: ./unused-reference-link-1

Generating the README:

$ python -m mdreftidy.cli ./mdreftidy/examples/SIMPLE.md --move-to-bottom --remove-unused --sort-ref-blocks --renumber -o - 2>/dev/null
# Example markdown file

## Reference link: Out of order

[Reference link: Out of order 1][1], [Reference link: Out of order 2][2].


## Footnotes

[1]: ./reference-link-out-of-order-3
[2]: ./reference-link-out-of-order-2

All together now:

Output of `bash ./snipinator/examples/simple_example.sh`

💻 Command Line Options

Output of `python -m mdreftidy.cli --help`

💡 Examples

✅ Requirements

  • Linux-like environment
    • Why: Uses pexpect.spawn().
  • Python 3.8+
    • Why: Some dev dependencies require Python 3.8+.

Tested Platforms

  • WSL2 Ubuntu 20.04, Python 3.8.0.
  • Ubuntu 20.04, Python 3.8.0, 3.9.0, 3.10.0, 3.11.0, 3.12.0, tested in GitHub Actions workflow (build-and-test.yml).

🐳 Docker Image

Docker images are published to ghcr.io/realazthat/mdreftidy at each tag.

# View the template file.
cat "mdreftidy/examples/SIMPLE.md"

# Use the published images at ghcr.io/realazthat/mdreftidy.
# /data in the docker image is the working directory, so paths are simpler.
docker run --rm --tty \
  -v "${PWD}:/data" \
  ghcr.io/realazthat/mdreftidy:v0.6.1 \
  "mdreftidy/examples/SIMPLE.md" \
  -o "mdreftidy/examples/SIMPLE.tidied.md" \
  --move-to-bottom --remove-unused --sort-ref-blocks --renumber

# Now --check to verify:
docker run --rm --tty \
  -v "${PWD}:/data" \
  ghcr.io/realazthat/mdreftidy:v0.6.1 \
  --check \
  "mdreftidy/examples/SIMPLE.tidied.md" \
  --move-to-bottom --remove-unused --sort-ref-blocks --renumber

# View the remotified file.
cat "mdreftidy/examples/SIMPLE.tidied.md"

If you want to build the image yourself, you can use the Dockerfile in the repository.

docker build -t my-mdreftidy-image .

# View the template file.
cat "mdreftidy/examples/SIMPLE.md"

# /data in the docker image is the working directory, so paths are simpler.
docker run --rm --tty \
  -v "${PWD}:/data" \
  my-mdreftidy-image \
  "mdreftidy/examples/SIMPLE.md" \
  -o "mdreftidy/examples/SIMPLE.tidied.md" \
  --move-to-bottom --remove-unused --sort-ref-blocks --renumber

# Now --check to verify:
docker run --rm --tty \
  -v "${PWD}:/data" \
  my-mdreftidy-image \
  --check \
  "mdreftidy/examples/SIMPLE.tidied.md" \
  --move-to-bottom --remove-unused --sort-ref-blocks --renumber

# View the remotified file.
cat "mdreftidy/examples/SIMPLE.tidied.md"

🤏 Versioning

We use SemVer for versioning. For the versions available, see the tags on this repository.

🔑 License

This project is licensed under the MIT License - see the ./LICENSE.md file for details.

🙏 Thanks

Main libraries used in mdreftidy are:

🤝 Related Projects

Not complete, and not necessarily up to date. Make a PR (contributions) to insert/modify.

Project Stars Last Update Language Platform Similarity X Obviousness
dce/mdrenum 2 2023/11/16 JS CLI ⭐⭐⭐⭐⭐
unified-utils 4 JS remark/CLI ⭐⭐⭐⭐⭐

🫡 Contributions

Development environment: Linux-like

  • For running pre.sh (Linux-like environment).

    • From ./.github/dependencies.yml, which is used for the GH Action to do a fresh install of everything:

      bash: scripts.
      findutils: scripts.
      grep: tests.
      xxd: tests.
      git: scripts, tests.
      xxhash: scripts (changeguard).
      rsync: out-of-directory test.
      expect: for `unbuffer`, useful to grab and compare ansi color symbols.
      jq: dependency for [yq](https://github.com/kislyuk/yq), which is used to generate
        the README; the README generator needs to use `tomlq` (which is a part of `yq`)
        to query `pyproject.toml`.
      
    • Requires pyenv, or an exact matching version of python as in .python-version (which is currently 3.8.0 ).

    • jq, (installation) required for yq, which is itself required for our ./README.md generation, which uses tomlq (from the yq package) to include version strings from ./pyproject.toml.

    • act (to run the GH Action locally):

      • Requires nodejs.
      • Requires Go.
      • docker.
    • Generate animation:

      • docker
    • docker (for building the docker image).

Commit Process

  1. (Optionally) Fork the develop branch.
  2. Stage your files: git add path/to/file.py.
  3. bash ./scripts/pre.sh, this will format, lint, and test the code.
  4. git status check if anything changed (generated ./README.md for example), if so, git add the changes, and go back to the previous step.
  5. git commit -m "...".
  6. Make a PR to develop (or push to develop if you have the rights).

🔄🚀 Release Process

These instructions are for maintainers of the project.

  1. In the develop branch, run bash ./scripts/pre.sh to ensure everything is in order.
  2. In the develop branch, bump the version in ./pyproject.toml, following semantic versioning principles. Also modify the last_release and last_stable_release in the [tool.mdreftidy-project-metadata] table as appropriate. Run bash ./scripts/pre.sh to ensure everything is in order.
  3. In the develop branch, commit these changes with a message like "Prepare release X.Y.Z". (See the contributions section above).
  4. Merge the develop branch into the master branch: git checkout master && git merge develop --no-ff.
  5. master branch: Tag the release: Create a git tag for the release with git tag -a vX.Y.Z -m "Version X.Y.Z".
  6. Publish to PyPI: Publish the release to PyPI with bash ./scripts/deploy-to-pypi.sh.
  7. Push to GitHub: Push the commit and tags to GitHub with git push && git push --tags.
  8. The --no-ff option adds a commit to the master branch for the merge, so refork the develop branch from the master branch: git checkout develop && git merge master.
  9. Push the develop branch to GitHub: git push origin develop.