Create Service

Create a service to expose the LLaMA deployment. If you are using a Linux operating system, then run the following syntax to create the service.yaml file.

nano service.yaml

If you are using a Windows operating system, open a text editor such as Notepad or Notepad++.

Text Editor

Enter the following syntax.

apiVersion: v1
kind: Service
metadata:
  name: llama-31-70b-instruct
  namespace: vllm
  annotations:
    metallb.universe.tf/address-pool: address-pool
spec:
  ports:
  - name: http-vllm
    port: 80
    protocol: TCP
    targetPort: 8000
  selector:
    app: llama-31-70b-instruct
  sessionAffinity: None
  type: LoadBalancer

If you are using a Linux operating system, run the following syntax but If you are using a Windows operating system, after save the file as secret.yaml, in CMD navigate to the folder that contains the secret.yaml file and run the following syntax.

kubectl apply -f service.yaml

Success Apply service.yaml

To delete the secret.yaml configuration that has been applied, run the following syntax.

kubectl delete -f secret.yaml -n [namespace]

Replace [namespace] with the namespace you created in the sub-chapter Create Namespace.