Jeremy Howard jph00

Introduction

Minai is a flexible and lightweight deep learning training framework developed interactively in the part 2 of fast.ai's 2022-23 course.

The core of minai is the Learner class, which orchestrates the training process through an extensive callback system, allowing users to easily modify or extend any part of the training loop.

Minai provides a set of utilities for data handling, device management, visualization, and performance optimization, making it suitable for both quick prototyping and advanced deep learning projects.

README.md

Hypermedia Systems: book summary

This is a summary of "Hypermedia Systems" by Carson Gross, Adam Stepinski, Deniz Akşimşek. Summary created by Jeremy Howard with AI help.

Website for the book
The revolutionary ideas that empowered the Web
A simpler approach to building applications on the Web and beyond with htmx and Hyperview
Enhancing web applications without using SPA frameworks

1: Hypermedia: A Reintroduction

Command Mode (press `Esc` to enable)

F : find and replace

↩ : enter edit mode

⌘⇧F : open the command palette

⌘⇧P : open the command palette

	#!/usr/bin/env python3
	# audio_data2url.py
	# Author: Scott H. Hawley
	# License: MIT
	# Date: Sep 2, 2024

	# Description: This script will convert base64 audio src data in a Jupyter notebook to URLs of the same audio
	# which is saved in a separate branch of the same GitHub repository. The script will save the audio files in a
	# directory named 'audio_files' and commit them to the 'audio-storage' branch. The script will then replace the
	# base64 data with the raw URL of the audio file in the notebook. The script can be run on a single notebook file

	<html>
	<head>
	<script src="https://gnat.github.io/css-scope-inline/script.js"></script>
	<script src="https://gnat.github.io/surreal/surreal.js"></script>
	</head>
	<body>
	<style>
	* { box-sizing: border-box; }
	html, body { width: 100%; height: 100%; display: flex; justify-content: center; align-items: center; }
	body { font-family: system-ui, sans-serif; perspective: 1500px; background: linear-gradient(#666, #222); }

	# Sketch-specific note: a roughly ~25 run battery for this code estimated a roughly ~93.11% accuracy in the same number of steps as the baseline network, ~1.7x runtime overhead (much of which goes to the torch.randn allocations and extra layer calculations).

	# Note: The one change we need to make if we're in Colab is to uncomment this below block.
	# If we are in an ipython session or a notebook, clear the state to avoid bugs
	#"""
	try:
	_ = get_ipython().__class__.__name__
	## we set -f below to avoid prompting the user before clearing the notebook state
	%reset -f
	except NameError:

	"""
	Testing flash attn with multipacking which essentially packs sequences using https://github.com/imoneoi/multipack_sampler,
	and passes a single sequence of `1 x (bs x seqlen)` to the model to avoid padding.

	An alternative is to use block diagonal attention as attention bias, but the following uses flash attention 2 which
	is much faster.

	Multipacking can be used to speed up both pretraining and finetuning.
	"""

	import os
	import openai
	from jinja2 import Template, meta, Environment
	from dotenv import load_dotenv
	load_dotenv() # add a .env file with the following
	# setup is for azure, change accordingly for normal openai
	openai.api_key = os.getenv("OPENAI_API_KEY")
	openai.api_type = os.getenv("OPENAI_API_TYPE")
	openai.api_version = os.getenv("OPENAI_API_VERSION")
	openai.api_base = os.getenv("OPENAI_API_BASE")

	import argparse
	import copy

	import torch

	import datasets as hfds
	import transformers

	from tqdm.auto import tqdm
	import wandb