CLI Reference

The following documentation is auto-generated from the ilab code.

ilab

CLI for interacting with InstructLab.

If this is your first time running ilab, it’s best to start with ilab config init to create the environment.

ilab [OPTIONS] COMMAND [ARGS]...

Options

--config <config_file>

Path to a configuration file.

Default:

'/home/runner/.config/instructlab/config.yaml'

-v, --verbose

Enable debug logging (repeat for even more verbosity)

--version

Show the version and exit.

config

Command Group for Interacting with the Config of InstructLab.

If this is your first time running ilab, it’s best to start with ilab config init to create the environment.

ilab config [OPTIONS] COMMAND [ARGS]...

edit

Launch $EDITOR to edit the configuration file.

ilab config edit [OPTIONS]

init

Initializes environment for InstructLab

ilab config init [OPTIONS]

Options

--interactive, --non-interactive

Initialize the environment assuming defaults.

Default:

True

--model-path <model_path>

Path to the model used during generation.

Default:

"The instructlab data files location per the user's system."

--taxonomy-base <taxonomy_base>

Base git-ref to use when listing/generating new taxonomy.

Default:

'origin/main'

--taxonomy-path <taxonomy_path>

Path to where the taxonomy should be cloned.

Default:

"The instructlab data files location per the user's system."

--repository <repository>

Taxonomy repository location.

Default:

'https://github.com/instructlab/taxonomy.git'

--min-taxonomy

Shallow clone the taxonomy repository with minimum size. Please do not use this option if you are planning to contribute back using the same taxonomy repository.

--profile <profile>

Overwrite the default values in the generated config.yaml by passing in an existing configuration yaml.

--config <config_file>

Path to a configuration file.

Environment variables

ILAB_GLOBAL_CONFIG

Provide a default for --config

show

Displays the current config as YAML with an option to extract a specific key.

ilab config show [OPTIONS]

Options

-k, --key <key_path>

Show only a specific section of the configuration, e.g., -k chat or -k chat.context

-wc, --without-comments

Show the config without comments. Can be used in conjunction with -k.

data

Command Group for Interacting with the Data generated by InstructLab.

If this is your first time running ilab, it’s best to start with ilab config init to create the environment.

ilab data [OPTIONS] COMMAND [ARGS]...

generate

Generates synthetic data to enhance your example data

ilab data generate [OPTIONS]

Options

--model <model_path>

--num-cpus <num_cpus>

Number of processes to use.

Default:

10

--chunk-word-count <chunk_word_count>

Number of words to chunk the document

Default:

1000

--sdg-scale-factor <sdg_scale_factor>

Number of instructions to generate for each seed example. The examples map to sample q&a pairs for new skills. For knowledge, examples are generated with both the sample q&a pairs, as well as chunks of the knowledge document(s), so the resulting data set is typically larger for a knowledge addition for the same value of –sdg-scale-factor.

--taxonomy-path <taxonomy_path>

--taxonomy-base <taxonomy_base>

--output-dir <output_dir>

--quiet

Suppress output of synthesized instructions.

--endpoint-url <endpoint_url>

Custom URL endpoint for OpenAI-compatible API. Defaults to the ilab model serve endpoint.

--api-key <api_key>

API key for API endpoint. [default: config.DEFAULT_API_KEY]

--yaml-rules <yaml_rules>

Custom rules file for YAML linting.

--server-ctx-size <server_ctx_size>

The context size is the maximum number of tokens the server will consider.

Default:

4096

--tls-insecure

Disable TLS verification.

--tls-client-cert <tls_client_cert>

Path to the TLS client certificate to use.

Default:

''

--tls-client-key <tls_client_key>

Path to the TLS client key to use.

Default:

''

--tls-client-passwd <tls_client_passwd>

TLS client certificate password.

--model-family <model_family>

Force model family to use when picking a generation template

--pipeline <pipeline>

--batch-size <batch_size>

Number of elements to process in each batch through the SDG pipeline. Enabled by default for the vLLM serving backend, with a batch size of 8 chosen based on experiments to optimize for throughput. Use 0 to disable.

--enable-serving-output

Print serving engine logs.

--gpus <gpus>

--max-num-tokens <max_num_tokens>

list

lists datasets

ilab data list [OPTIONS]

Options

--dataset-dirs <dataset_dirs>

Base directories where datasets are stored.

Default:

'/home/runner/.local/share/instructlab/datasets'

model

Command Group for Interacting with the Models in InstructLab.

If this is your first time running ilab, it’s best to start with ilab config init to create the environment.

ilab model [OPTIONS] COMMAND [ARGS]...

chat

Runs a chat using the modified model

ilab model chat [OPTIONS] [QUESTION]...

Options

-m, --model <model>

Required

-c, --context <context>

Required

-s, --session <session>

-qq, --quick-question

Exit after answering question.

--max-tokens <max_tokens>

--endpoint-url <endpoint_url>

Custom URL endpoint for OpenAI-compatible API. Defaults to the ilab model serve endpoint.

--api-key <api_key>

API key for API endpoint. [default: config.DEFAULT_API_KEY]

--tls-insecure

Disable TLS verification.

--tls-client-cert <tls_client_cert>

Path to the TLS client certificate to use.

Default:

''

--tls-client-key <tls_client_key>

Path to the TLS client key to use.

Default:

''

--tls-client-passwd <tls_client_passwd>

TLS client certificate password.

--model-family <model_family>

Force model family to use when picking a chat template

--serving-log-file <serving_log_file>

Log file path to write server logs to.

-t, --temperature <temperature>

Arguments

QUESTION

Optional argument(s)

convert

Converts model to GGUF

ilab model convert [OPTIONS]

Options

--model-dir <model_dir>

Required Base directory where model is stored.

--adapter-file <adapter_file>

LoRA adapter to fuse.

-sd, --skip-de-quantize

Skip de-quantization.

-sq, --skip-quantize

Whether to skip quantization while converting to GGUF.

--model-name <model_name>

Name of the model being trained/converted. Informs the naming of the final trained model file

download

Downloads model from a specified repository

ilab model download [OPTIONS]

Options

-rp, --repository <repositories>

Hugging Face or OCI repository of the model to download.

Default:

'instructlab/granite-7b-lab-GGUF', 'instructlab/merlinite-7b-lab-GGUF', 'TheBloke/Mistral-7B-Instruct-v0.2-GGUF'

-rl, --release <releases>

The revision of the model to download - e.g. a branch, tag, or commit hash for Hugging Face repositories and tag or commit hash for OCI repositories.

Default:

'main', 'main', 'main'

--filename <filenames>

Name of the model file to download from the Hugging Face repository.

Default:

'The default model location in the instructlab data directory.'

--model-dir <model_dir>

The local directory to download the model files into.

Default:

'The default system model location store, located in the data directory.'

--hf-token <hf_token>

User access token for connecting to the Hugging Face Hub.

Environment variables

HF_TOKEN

Provide a default for --hf-token

evaluate

Evaluates a trained model

ilab model evaluate [OPTIONS]

Options

--model <model>

--base-model <base_model>

Required

--benchmark <benchmark>

Required Benchmarks to run during evaluation

Options:

mmlu | mmlu_branch | mt_bench | mt_bench_branch

--judge-model <judge_model>

--output-dir <output_dir>

--max-workers <max_workers>

--taxonomy-path <taxonomy_path>

--branch <branch>

--base-branch <base_branch>

--few-shots <few_shots>

--batch-size <batch_size>

--tasks-dir <tasks_dir>

--gpus <gpus>

Number of GPUs to utilize for evaluation (not applicable to llama-cpp)

--merge-system-user-message

Indicates whether to merge system and user message for mt_bench and mt_bench_branch (required for Mistral based judges)

--backend <backend>

Serving backend to use for the model and base model (if applicable) during evaluation. Options are vllm and llama-cpp.

Options:

llama-cpp | vllm

--judge-backend <judge_backend>

Serving backend to use for the judge model for during mt_bench or mt_bench_branch evaluation. Options are vllm and llama-cpp.

Options:

llama-cpp | vllm

--tls-insecure

Disable TLS verification for model serving.

--tls-client-cert <tls_client_cert>

Path to the TLS client certificate to use for model serving.

Default:

''

--tls-client-key <tls_client_key>

Path to the TLS client key to use for model serving.

Default:

''

--tls-client-passwd <tls_client_passwd>

TLS client certificate password for model serving.

--enable-serving-output

Print serving engine logs.

list

Lists models

ilab model list [OPTIONS]

Options

--model-dirs <model_dirs>

Base directories where models are stored.

Default:

'/home/runner/.cache/instructlab/models'

--list-checkpoints

Also list checkpoints [in addition to existing models].

serve

Starts a local server

ilab model serve [OPTIONS]

Options

--model-path <model_path>

Required

--gpu-layers <gpu_layers>

Required

--num-threads <num_threads>

The number of CPU threads to use.

--max-ctx-size <max_ctx_size>

--model-family <model_family>

Model family is used to specify which chat template to serve with

--log-file <log_file>

Log file path to write server logs to.

--chat-template <chat_template>

--backend <backend>

Options:

llama-cpp | vllm

--gpus <gpus>

-h, --host <host>

-p, --port <port>

test

Runs basic test to ensure model correctness

ilab model test [OPTIONS]

Options

--data-dir <data_dir>

Base directory where data is stored.

Default:

'Default internal data directory, stored in the instructlab package.'

--model-dir <model_dir>

Base directory where model is stored.

Default:

'Default instructlab system checkpoints directory.'

--adapter-file <adapter_file>

LoRA adapter to use for test. Set to ‘None’ to force only testing behavior from before training.

Default:

'auto'

-m, --model <model>

Base model name to test on Linux

Default:

<function <lambda> at 0x7fca5702bf60>

-t, --test_file <test_file>

Test data file

--api-key <api_key>

API key for API endpoint. [default: cfg.DEFAULT_API_KEY]

--model-family <model_family>

Force model family to use when picking a generation template

train

Takes synthetic data generated locally with ilab data generate and the previous model and learns a new model using the MLX API. On success, writes newly learned model to {model_dir}/mlx_model, which is where chatmlx will look for a model.

ilab model train [OPTIONS]

Options

--data-path <data_path>

Required

--ckpt-output-dir <ckpt_output_dir>

Required

--data-output-dir <data_output_dir>

Required

--input-dir <input_dir>

Path to generated files to use as input.

--gguf-model-path <gguf_model_path>

Local directory where gguf model is stored.

--skip-preprocessing

Skips data preprocessing step for MLX training if data is already cached.

--tokenizer-dir <tokenizer_dir>

Base directory where tokenizer is stored.

--model-path <model_path>

Required

--iters <iters>

Number of iterations to train LoRA.

--local

Whether or not model_dir is remote from HuggingFace.

-sq, --skip-quantize

Whether to skip quantization while converting to MLX. This parameter will be ignored if –gguf-model-path and –tokenizer-dir are specified.

--num-epochs <num_epochs>

Required

--device <device>

Options:

cpu | cuda | hpu | mps

--max-seq-len <max_seq_len>

Required

--max-batch-len <max_batch_len>

Required

--effective-batch-size <effective_batch_size>

Required

--save-samples <save_samples>

Required

--learning-rate <learning_rate>

Required

--warmup-steps <warmup_steps>

Required

--deepspeed-cpu-offload-optimizer <deepspeed_cpu_offload_optimizer>

Required

--deepspeed-cpu-offload-optimizer-ratio <deepspeed_cpu_offload_optimizer_ratio>

Required

--deepspeed-cpu-offload-optimizer-pin-memory <deepspeed_cpu_offload_optimizer_pin_memory>

Required

--fsdp-cpu-offload-optimizer <fsdp_cpu_offload_optimizer>

--distributed-backend <distributed_backend>

--lora-rank <lora_rank>

--lora-alpha <lora_alpha>

--lora-dropout <lora_dropout>

--lora-target-modules <lora_target_modules>

--lora-quantize-dtype <lora_quantize_dtype>

--is-padding-free <is_padding_free>

--use-dolomite <use_dolomite>

Required

--gpus <nproc_per_node>

--nnodes <nnodes>

Required

--node-rank <node_rank>

Required

--rdzv-id <rdzv_id>

Required

--rdzv-endpoint <rdzv_endpoint>

Required

--disable-flash-attn

--strategy <strategy>

If chosen, will run the selected training strategy instead of a single training run.

Options:

lab-multiphase | lab-skills-only

--phased-base-dir <phased_base_dir>

--phased-phase1-data <phased_phase1_data>

Path to .jsonl file that will be used for the first phase of end-to-end training.

--phased-phase1-num-epochs <phased_phase1_num_epochs>

--phased-phase1-samples-per-save <phased_phase1_samples_per_save>

--phased-phase1-learning-rate <phased_phase1_learning_rate>

--phased-phase1-effective-batch-size <phased_phase1_effective_batch_size>

--phased-phase2-data <phased_phase2_data>

Path to .jsonl file that will be used for the second phase of end-to-end training.

--phased-phase2-num-epochs <phased_phase2_num_epochs>

--phased-phase2-samples-per-save <phased_phase2_samples_per_save>

--phased-phase2-learning-rate <phased_phase2_learning_rate>

--phased-phase2-effective-batch-size <phased_phase2_effective_batch_size>

--phased-mt-bench-judge <phased_mt_bench_judge>

-y, --skip-user-confirm

Skips any user confirmation prompts.

--enable-serving-output

Print serving engine logs during phased training checkpoint evaluation.

--checkpoint-at-epoch

By default, checkpoints are saved at the end of each training epoch. This option disables this behavior.

--pipeline <pipeline>

Options:

simple | full | accelerated

--training-journal <training_journal>

--force-clear-phased-cache

Clear phased cache (journal, checkpoints, metadata). Helpful paired with ‘–skip-user-confirm’

--optimize-memory

Optimize Memory Usage on CPU and MacOS. This uses the torch_dtype=’auto’ instead of float32

upload

Uploads model to a specified location

ilab model upload [OPTIONS]

Options

-m, --model <model>

Required Path to the model to upload or name of an existing checkpoint.

--dest-type <dest_type>

The type of destination to upload to - can be ‘hf’, ‘oci’, or ‘s3’

Options:

oci | s3 | hf

-d, --destination <destination>

Required Destination for the model to be uploaded to. Ex: for a Hugging Face repo, should be ‘<username>/<repo-name>’

-rl, --release <release>

The revision to upload the model to - e.g. a branch for Hugging Face repositories.

Default:

'main'

--hf-token <hf_token>

User access token for connecting to the Hugging Face Hub.

Environment variables

HF_TOKEN

Provide a default for --hf-token

system

Command group for all system-related command calls

ilab system [OPTIONS] COMMAND [ARGS]...

info

Print system information

ilab system info [OPTIONS]

taxonomy

Command Group for Interacting with the Taxonomy of InstructLab.

If this is your first time running ilab, it’s best to start with ilab config init to create the environment.

ilab taxonomy [OPTIONS] COMMAND [ARGS]...

diff

Lists taxonomy files that have changed since <taxonomy-base> and checks that taxonomy is valid. Similar to ‘git diff <ref>’.

ilab taxonomy diff [OPTIONS]

Options

--taxonomy-path <taxonomy_path>

Path to where the taxonomy is stored locally.

Default:

'Default taxonomy location in the instructlab data directory.'

--taxonomy-base <taxonomy_base>

Base git-ref to use for taxonomy, use ‘empty’ for the entire repo contents.

--yaml-rules <yaml_rules>

Custom rules file for YAML linting.

--quiet

Suppress all output. Call returns 0 if check passes, 1 otherwise.