Install BigDL-Nano in Google Colab#
This page is still a work in progress.
In this guide, we will show you how to install BigDL-Nano in Google Colab, and the solutions to possible version conflicts caused by pre-installed packages in Colab hosted runtime.
Please select the corresponding section to follow for your specific usage.
For PyTorch users, you need to install BigDL-Nano for PyTorch first:
!pip install bigdl-nano[pytorch]
!pip install --pre --upgrade bigdl-nano[pytorch]
For Google Colab hosted runtime,
source bigdl-nano-init is hardly to take effect as environment variables need to be set before jupyter kernel is started.
To avoid version conflicts caused by
torchtext, you should uninstall it:
!pip uninstall -y torchtext
To enable ONNXRuntime acceleration, you need to install corresponding onnx packages:
!pip install onnx onnxruntime
OpenVINO / Post-training Optimization Tools (POT)#
To enable OpenVINO acceleration, or use POT for quantization, you need to install the OpenVINO toolkit:
!pip install openvino-dev # Please remember to restart runtime to use packages with newly-installed version
If you meet
ValueError: numpy.ndarray size changed, may indicate binary incompatibility. Expected 88 from C header, got 80 from PyObject when using
InferenceOptimizer.quantize function, you could try to solve it by upgrading
!pip install --upgrade numpy # Please remember to restart runtime to use numpy with newly-installed version
Intel Neural Compressor (INC)#
To use INC as your quantization backend, you need to install it:
!pip install neural-compressor==1.11.0
!pip install neural-compressor==1.11.0 onnx onnxruntime onnxruntime_extensions