[AI] LLM RAG 환경 구성 - 1. Ollama 설치
페이지 정보
본문
안녕하세요.
꿈꾸는여행자입니다.
LLM RAG 환경 구성을 다루고자 합니다.
이번 항목에서는 Ollama 설치 까지 실습을 진행해 보도록 하겠습니다.
상세 내역은 아래와 같습니다.
감사합니다.
> 아래
________________
목차
1. Overview 3
1.1. 분석 3
1.1.1. 관련 검색 3
1.1.2. Url 3
1.2. 설계 3
2. Ollama 5
2.1. Prepare 5
2.1.1. Setting - sudo user 5
2.1.2. Package 5
2.1.3. cuda 9
2.2. Install Ollama 11
2.2.1. Download and Install Ollama 11
2.2.2. Install Ollama Web-UI [Options] 11
2.3. HuggingFace-Hub 12
2.3.1. HuggingFace-Hub 설치 12
2.3.2. Download GGUF 파일 13
2.3.3. Modelfile 준비 14
2.4. Run Ollama 15
2.4.1. List 15
2.4.2. Create 15
2.4.3. Run 16
3. LangServe 17
3.1. Prepare 17
3.1.1. git clone 17
3.1.2. Install Python Packages 17
3.1.3. Check Source 18
3.2. LangServe에서 Ollama 체인 생성 19
3.2.1. Run - langserve 19
3.3. Check - Browser 19
3.3.1. LangServe 19
3.3.2. Fast API 20
4. streamlit 21
4.1. Prepare 21
4.1.1. Install Python Packages 21
4.2. Setting 21
4.2.1. main.py - LANGSERVE_ENDPOINT 21
4.3. Run 22
4.3.1. Run - streamlit 22
4.3. Check - Browser 23
________________
1. Overview
1.1. 분석
1.1.1. 관련 검색
* Word
* ollama langserve streamlit
1.1.2. Url
* RAG
* 무료로 한국어 파인튜닝 모델 받아서 나만의 로컬 LLM 호스팅 하기(LangServe) + RAG 까지!!
* https://github.com/teddylee777/langserve_ollama
* https://www.youtube.com/watch?v=VkcaigvTrug
* BEST
* Ollama
* 로컬에서 무료로 사용할 수 있는 LLM 도구, Ollama 활용 가이드
* https://anpigon.tistory.com/434
* Linux에서 Ollama 기반으로 LLM 실행하기
* https://velog.io/@quanternoh/Ollama-%EA%B8%B0%EB%B0%98%EC%9D%98-CodeLlama-%EB%A1%9C%EC%BB%AC-%EC%8B%A4%ED%96%89
* Ollama 를 활용해서 개인형 LLM 서버 구성하기
* https://devocean.sk.com/blog/techBoardDetail.do?ID=165685
* [LLM] Ollama Web-UI 설치
* https://pasus.tistory.com/322
* Amazon EC2에서 Ollama 서버 및 Ollama 웹 UI를 사용하여 로컬 LLM을 배포하는 방법
* https://medium.com/@amardewri/how-to-deploy-local-llm-using-ollama-server-and-ollama-web-ui-on-amazon-ec2-91ffee49d9cc
* Docker 기반
* Cuda
* Installing CUDA on Rocky Linux
* https://doc.flowhub.com.cn/en/FAQ/RockylinuxInstallCuda/
1.2. 설계
________________
2. Ollama
2.1. Prepare
2.1.1. Setting - sudo user
* 대상 계정에 sudo 권한 설정
* usermod -aG wheel lds
su - lds
[root@llm ~]# cat /etc/group | grep wheel
wheel:x:10:
[root@llm ~]# usermod -aG wheel lds
[root@llm ~]# cat /etc/group | grep wheel
wheel:x:10:lds
[root@llm ~]# su - lds
[lds@llm ~]$ sudo ls -la /root
[sudo] password for lds:
total 36
dr-xr-x---. 5 root root 4096 May 2 17:23 .
dr-xr-xr-x. 18 root root 235 Jan 17 14:38 ..
-rw-------. 1 root root 1023 Jan 17 14:26 anaconda-ks.cfg
-rw-------. 1 root root 512 May 2 17:21 .bash_history
-rw-r--r--. 1 root root 18 May 11 2022 .bash_logout
-rw-r--r--. 1 root root 141 May 11 2022 .bash_profile
-rw-r--r--. 1 root root 429 May 11 2022 .bashrc
drwx------. 3 root root 17 Apr 3 09:51 .cache
-rw-r--r--. 1 root root 100 May 11 2022 .cshrc
drwxr-xr-x. 6 root root 87 Apr 3 11:31 llama2env
drwx------. 2 root root 6 Jan 17 14:19 .ssh
-rw-r--r--. 1 root root 129 May 11 2022 .tcshrc
-rw-------. 1 root root 1358 May 2 17:23 .viminfo
[lds@llm ~]$
*
2.1.2. Package
* Install git
* git 설치
sudo dnf install git git-core
[root@llm ~]# id
uid=0(root) gid=0(root) groups=0(root) context=unconfined_u:unconfined_r:unconfined_t:s0-s0:c0.c1023
[root@llm ~]# dnf install git git-core
Last metadata expiration check: 0:06:24 ago on Thu 02 May 2024 05:09:38 PM KST.
Dependencies resolved.
======================================================================================================================
Package Architecture Version Repository Size
======================================================================================================================
Installing:
git x86_64 2.39.3-1.el9_2 appstream 61 k
git-core x86_64 2.39.3-1.el9_2 appstream 4.2 M
Installing dependencies:
git-core-doc noarch 2.39.3-1.el9_2 appstream 2.6 M
perl-Error noarch 1:0.17029-7.el9 appstream 41 k
perl-Git noarch 2.39.3-1.el9_2 appstream 37 k
perl-TermReadKey x86_64 2.38-11.el9 appstream 36 k
perl-lib x86_64 0.65-480.el9 appstream 15 k
Transaction Summary
======================================================================================================================
Install 7 Packages
Total download size: 7.0 M
Installed size: 36 M
Is this ok [y/N]: y
Downloading Packages:
(1/7): perl-lib-0.65-480.el9.x86_64.rpm 76 kB/s | 15 kB 00:00
(2/7): perl-Error-0.17029-7.el9.noarch.rpm 202 kB/s | 41 kB 00:00
(3/7): perl-Git-2.39.3-1.el9_2.noarch.rpm 933 kB/s | 37 kB 00:00
(4/7): perl-TermReadKey-2.38-11.el9.x86_64.rpm 66 kB/s | 36 kB 00:00
(5/7): git-2.39.3-1.el9_2.x86_64.rpm 253 kB/s | 61 kB 00:00
(6/7): git-core-doc-2.39.3-1.el9_2.noarch.rpm 1.7 MB/s | 2.6 MB 00:01
(7/7): git-core-2.39.3-1.el9_2.x86_64.rpm 1.5 MB/s | 4.2 MB 00:02
----------------------------------------------------------------------------------------------------------------------
Total 1.5 MB/s | 7.0 MB 00:04
Running transaction check
Transaction check succeeded.
Running transaction test
Transaction test succeeded.
Running transaction
Preparing : 1/1
Installing : git-core-2.39.3-1.el9_2.x86_64 1/7
Installing : git-core-doc-2.39.3-1.el9_2.noarch 2/7
Installing : perl-TermReadKey-2.38-11.el9.x86_64 3/7
Installing : perl-lib-0.65-480.el9.x86_64 4/7
Installing : perl-Error-1:0.17029-7.el9.noarch 5/7
Installing : git-2.39.3-1.el9_2.x86_64 6/7
Installing : perl-Git-2.39.3-1.el9_2.noarch 7/7
Running scriptlet: perl-Git-2.39.3-1.el9_2.noarch 7/7
Verifying : perl-Error-1:0.17029-7.el9.noarch 1/7
Verifying : perl-lib-0.65-480.el9.x86_64 2/7
Verifying : perl-TermReadKey-2.38-11.el9.x86_64 3/7
Verifying : perl-Git-2.39.3-1.el9_2.noarch 4/7
Verifying : git-core-doc-2.39.3-1.el9_2.noarch 5/7
Verifying : git-core-2.39.3-1.el9_2.x86_64 6/7
Verifying : git-2.39.3-1.el9_2.x86_64 7/7
Installed:
git-2.39.3-1.el9_2.x86_64 git-core-2.39.3-1.el9_2.x86_64 git-core-doc-2.39.3-1.el9_2.noarch
perl-Error-1:0.17029-7.el9.noarch perl-Git-2.39.3-1.el9_2.noarch perl-TermReadKey-2.38-11.el9.x86_64
perl-lib-0.65-480.el9.x86_64
Complete!
[root@llm ~]#
*
* python
* python과 pip 확인
sudo dnf install python3.11
python3.11 --version
pip --version
[lds@llm ~]$ sudo dnf install python3.11
Last metadata expiration check: 0:41:48 ago on Thu 02 May 2024 05:09:38 PM KST.
Dependencies resolved.
======================================================================================================================
Package Architecture Version Repository Size
======================================================================================================================
Installing:
python3.11 x86_64 3.11.5-1.el9_3 appstream 26 k
Installing dependencies:
libnsl2 x86_64 2.0.0-1.el9.0.1 appstream 30 k
mpdecimal x86_64 2.5.1-3.el9 appstream 85 k
python3.11-libs x86_64 3.11.5-1.el9_3 appstream 9.3 M
python3.11-pip-wheel noarch 22.3.1-4.el9_3.1 appstream 1.4 M
python3.11-setuptools-wheel noarch 65.5.1-2.el9 appstream 712 k
Transaction Summary
======================================================================================================================
Install 6 Packages
Total download size: 12 M
Installed size: 47 M
Is this ok [y/N]: y
Downloading Packages:
(1/6): libnsl2-2.0.0-1.el9.0.1.x86_64.rpm 179 kB/s | 30 kB 00:00
(2/6): python3.11-3.11.5-1.el9_3.x86_64.rpm 79 kB/s | 26 kB 00:00
(3/6): mpdecimal-2.5.1-3.el9.x86_64.rpm 173 kB/s | 85 kB 00:00
(4/6): python3.11-setuptools-wheel-65.5.1-2.el9.noarch.rpm 617 kB/s | 712 kB 00:01
(5/6): python3.11-pip-wheel-22.3.1-4.el9_3.1.noarch.rpm 850 kB/s | 1.4 MB 00:01
(6/6): python3.11-libs-3.11.5-1.el9_3.x86_64.rpm 2.1 MB/s | 9.3 MB 00:04
----------------------------------------------------------------------------------------------------------------------
Total 1.9 MB/s | 12 MB 00:05
Running transaction check
Transaction check succeeded.
Running transaction test
Transaction test succeeded.
Running transaction
Preparing : 1/1
Installing : python3.11-pip-wheel-22.3.1-4.el9_3.1.noarch 1/6
Installing : python3.11-setuptools-wheel-65.5.1-2.el9.noarch 2/6
Installing : mpdecimal-2.5.1-3.el9.x86_64 3/6
Installing : libnsl2-2.0.0-1.el9.0.1.x86_64 4/6
Installing : python3.11-3.11.5-1.el9_3.x86_64 5/6
Installing : python3.11-libs-3.11.5-1.el9_3.x86_64 6/6
Running scriptlet: python3.11-libs-3.11.5-1.el9_3.x86_64 6/6
Verifying : libnsl2-2.0.0-1.el9.0.1.x86_64 1/6
Verifying : mpdecimal-2.5.1-3.el9.x86_64 2/6
Verifying : python3.11-libs-3.11.5-1.el9_3.x86_64 3/6
Verifying : python3.11-3.11.5-1.el9_3.x86_64 4/6
Verifying : python3.11-setuptools-wheel-65.5.1-2.el9.noarch 5/6
Verifying : python3.11-pip-wheel-22.3.1-4.el9_3.1.noarch 6/6
Installed:
libnsl2-2.0.0-1.el9.0.1.x86_64 mpdecimal-2.5.1-3.el9.x86_64
python3.11-3.11.5-1.el9_3.x86_64 python3.11-libs-3.11.5-1.el9_3.x86_64
python3.11-pip-wheel-22.3.1-4.el9_3.1.noarch python3.11-setuptools-wheel-65.5.1-2.el9.noarch
Complete!
[lds@llm ~]$
[lds@llm ~]$ python3.11 --version
Python 3.11.5
[lds@llm ~]$
[lds@llm ~]$ pip --version
pip 21.2.3 from /home/lds/Works/llm/env/lib64/python3.9/site-packages/pip (python 3.9)
[lds@llm ~]$
*
* python - env
* python 환경 구성 및 실행
* python3.11 로 실행
python3.11 -m ven env
source env/bin/activate
[lds@llm llm]$ pwd
/home/lds/Works/llm
[lds@llm llm]$ python3.11 -m venv env
[lds@llm llm]$ source env/bin/activate
(env) [lds@llm llm]$
*
2.1.3. cuda
https://doc.flowhub.com.cn/en/FAQ/RockylinuxInstallCuda/
Update the system version and kernel
sudo dnf update && sudo dnf upgrade -y
Install epel-release
sudo dnf install epel-release
Add the NVIDIA repository
sudo dnf config-manager --add-repo \
https://developer.download.nvidia.com/compute/cuda/repos/rhel8/x86_64/cuda-rhel8.repo
dnf clean all
dnf list
Install the corresponding development tools and header files
sudo dnf install kernel-devel-$(uname -r) kernel-headers-$(uname -r)
Install the NVIDIA driver
sudo dnf install nvidia-driver nvidia-settings
Install the CUDA driver
sudo dnf install cuda-driver
2.2. Install Ollama
2.2.1. Download and Install Ollama
* Install with one command:
* sudo 권한이 있는 User 또는 root로 실행
* 30분 소요
curl -fsSL https://ollama.com/install.sh | sh
[root@llm ~]# curl -fsSL https://ollama.com/install.sh | sh
>>> Downloading ollama...
######################################################################## 100.0%##O#-# ######################################################################## 100.0%
>>> Installing ollama to /usr/local/bin...
>>> Creating ollama user...
>>> Adding ollama user to render group...
>>> Adding ollama user to video group...
>>> Adding current user to ollama group...
>>> Creating ollama systemd service...
>>> Enabling and starting ollama service...
Created symlink /etc/systemd/system/default.target.wants/ollama.service → /etc/systemd/system/ollama.service.
>>> The Ollama API is now available at 127.0.0.1:11434.
>>> Install complete. Run "ollama" from the command line.
WARNING: No NVIDIA/AMD GPU detected. Ollama will run in CPU-only mode.
[root@llm ~]#
- 이전글[AI] LLM RAG 환경 구성 - 2 Ollama 실행 및 모델 등록 24.06.27
- 다음글딥 러닝 (Deep Learning) 21.08.04
댓글목록
등록된 댓글이 없습니다.