Kashif kashifulhaque

Aspect	On-Policy	Off-Policy
Definition	Learns the value function for the policy being used for action selection	Can learn about a different policy than the one being used for action selection
Policy Updating	Uses the same policy for both learning and action selection	Can use different policies for learning and action selection
Data Collection	Collects data using the current policy	Can use data collected from any policy
Exploration	Typically requires a balance between exploration and exploitation	Can learn from data collected u

In order to check similarity between 2 vectors, we can check the distance between them. There are a few different metrics to measure distance between 2 vectors.

import numpy as np

1. Cosine similarity

Measures the angle between 2 non-zero vectors. It ranges between $[-1, 1]$ $$cos (\theta) = \frac{\vec{p} \cdot \vec{q}}{||p|| \text{ } ||q||}$$

def cosine_similarity(p, q):

Add the unstable NixOS channel

sudo nix-channel --add https://nixos.org/channels/nixos-unstable nixos-unstable

Update the unstable NixOS channel

sudo nix-channel --update nixos-unstable

https://brev.dev/

https://vast.ai/

https://lambdalabs.com/service/gpu-cloud

https://www.paperspace.com/

https://huggingface.co/docs/hub/en/spaces-gpus

Ctrl + X kill active process
Ctrl + D exit active shell
Ctrl + L clear screen
Ctrl + Z put current process in background (fg to bring it back to foregorund)
Ctrl + A to go to front of a line
Ctrl + E to go the end of a line
Ctrl + F to go forward one character (right arrow)
Ctrl + B to go backward one character (left arrow)
Alt + F to go forward one word (vim w motion)
Alt + B to go backward one word (vim b motion)

To export and share your conda environment

conda activate your_env_name

conda env export > environment.yml

To import environment from YAML file

Neovim keybinds

Capital letters do the opposite of small letters in command (Press shift to trigger capital letters)
_ (underscore) to move the cursor at the beginning of line (doesn't switch to insert mode)
- 0 (zero) moves the cursor to the zeroth position of the line (doesn't switch to insert mode)
$ (dollar) to move the cursor at the end of line (doesn't switch to insert mode)
d$ will delete from wherever your cursor is till the end of the line
f<character> to move cursor to the first occurrence of <character>
- f( to move cursor to first occurence of (
t<character> to move cursor to upto but not on the first occurrence of <character>
t( to move cursor to first occurence of (

	import torch
	import numpy as np

	def check_gpu_capabilities():
	print("=== GPU Information ===")

	if not torch.cuda.is_available():
	print("❌ CUDA is not available on this system")
	return

	[shell]
	program = "wsl"
	args = [ "~" ]

	[font]
	size = 12
	[font.normal]
	family = "CodeNewRoman Nerd Font"
	style = "Regular"

	{
	"background": "#181818",
	"foreground": "p:white",
	"style": "powerline",
	"properties": {
	"display_mode": "environment",
	"fetch_virtual_env": true,
	"home_enabled": true
	},
	"type": "python",