To expose Redpanda (Kafka) externally in the United Manufacturing Hub (UMH), you need to adjust the Helm chart values accordingly. This guide provides step-by-step instructions for setting up external access to Redpanda in a single-node Kubernetes cluster.
Prerequisites
- Advanced User Access: You should have SSH access to the Kubernetes cluster or use tools like OpenLens to modify Helm chart values.
- Helm Installed: Ensure that Helm is installed on your local machine.
- Kubeconfig File: Access to the kubeconfig file (e.g.,
/etc/rancher/k3s/k3s.yaml). - Public IP Address: Replace
10.13.37.200with your actual public IP address.
Steps to Expose Redpanda Externally
1. Access Your Kubernetes Cluster
Log in to your Kubernetes cluster via SSH:
ssh user@your-cluster-ip
2. Update the UMH Helm Chart
Run the following Helm upgrade command to enable external access for Redpanda:
sudo helm upgrade united-manufacturing-hub united-manufacturing-hub/united-manufacturing-hub \
--namespace united-manufacturing-hub \
--kubeconfig /etc/rancher/k3s/k3s.yaml \
--set redpanda.external.enabled=true \
--set redpanda.external.service.enabled=true \
--set redpanda.external.type=LoadBalancer \
--set redpanda.tls.enabled=false \
--set "redpanda.external.addresses={YOUR_PUBLIC_IP}" \
--set redpanda.external.domain="" \
--wait
Note: Replace YOUR_PUBLIC_IP with your actual public IP address.
Explanation of the Command
--set redpanda.external.enabled=true: Enables external access for Redpanda.--set redpanda.external.service.enabled=true: Activates the external service.--set redpanda.external.type=LoadBalancer: Sets the service type toLoadBalancer.--set redpanda.tls.enabled=false: Disables TLS for simplicity (enable TLS in production environments).--set "redpanda.external.addresses={YOUR_PUBLIC_IP}": Specifies the external IP addresses.--set redpanda.external.domain="": Leaves the domain empty; useful if you're not using a custom domain.--wait: Waits until all resources are successfully upgraded.
3. Verify the External Service
Check if the Redpanda service is exposed externally:
kubectl --kubeconfig /etc/rancher/k3s/k3s.yaml get services -n united-manufacturing-hub
You should see a service of type LoadBalancer with an external IP assigned.
4. Restart Redpanda
Restart the Redpanda pod:
kubectl --kubeconfig /etc/rancher/k3s/k3s.yaml delete pod -n united-manufacturing-hub united-manufacturing-hub-kafka-0
4. Connect to Redpanda
You can now connect to Redpanda using the external IP and port 31092:
- Connection String:
YOUR_PUBLIC_IP:31092
Example Use Case: Kafka-to-Kafka Bridge
In scenarios like setting up a Kafka-to-Kafka bridge, you can use the above connection string to link your external Kafka clients to Redpanda.
5. Testing the Connection
Use a Kafka client to test the connection:
kafka-console-producer --broker-list YOUR_PUBLIC_IP:31092 --topic test-topic
Important Considerations
- Advertised Address: Failing to update the advertised address (
redpanda.external.addresses) will result in consumers/producers receiving internal cluster addresses, causing connection failures. - Security: Exposing services externally can introduce security risks. Ensure proper firewall rules and security measures are in place.
- TLS Encryption: TLS is disabled in this guide for simplicity. In a production environment, it is recommended to enable TLS for secure communication.
Additional Resources
- Redpanda Documentation: Expose Redpanda Outside Kubernetes Using a LoadBalancer
- United Manufacturing Hub Documentation
Troubleshooting
- Service Not Accessible: If the service isn't accessible externally, check firewall settings and ensure that the Kubernetes service has assigned the external IP.
- Connection Timeouts: Verify that the advertised addresses are correctly set and that there are no network policies blocking the traffic.
By following this guide, you should be able to expose Redpanda externally in the UMH and connect your external Kafka clients successfully.
