BigDL-LLM Installation: CPU#

Quick Installation#

Install BigDL-LLM for CPU supports using pip through:

pip install --pre --upgrade bigdl-llm[all] # install the latest bigdl-llm nightly build with 'all' option

Please refer to Environment Setup for more information.

Note

all option will trigger installation of all the dependencies for common LLM application development.

Important

bigdl-llm is tested with Python 3.9, which is recommended for best practices.

Environment Setup#

For optimal performance with LLM models using BigDL-LLM optimizations on Intel CPUs, here are some best practices for setting up environment:

First we recommend using Conda to create a python 3.9 enviroment:

conda create -n llm python=3.9
conda activate llm

pip install --pre --upgrade bigdl-llm[all] # install the latest bigdl-llm nightly build with 'all' option

Then for running a LLM model with BigDL-LLM optimizations (taking an example.py an example):

It is recommended to run directly with full utilization of all CPU cores:

python example.py