Deploy a Laravel Application to Kubernetes using Gitlab CI

Prerequisites

This article assumes you have a basic understanding of Docker and Kubernetes, Gitlab CI and that you have already set up a Kubernetes Cluster.

Start a Laravel Project

The first thing you’ll need is a Laravel application, use composer to start a new project.

Dockerize your Laravel Project

There’s a number of ways to dockerize your Laravel project. You may use the official Nginx and PHP images form dockerhub, but I found it’s a bit troublesome to set them up.

So instead of messing around with all the different kinds of docker images, I came across thecodingmachine/docker-images-php, a set of production-ready docker images.

To build a production-ready image, we will use thecodingmachine/php:7.3-v2-slim-apache as our base image. The Dockerfile looks like this:

We will configure Gitlab CI to build the docker image automatically later.

Create Kubernetes Deployment Files

Here are all the yaml files we need to deploy our Laravel Application.

Deployment

Out deployment.yaml contains two deployments actually. One is for our main Laravel application, while the other one is for Laravel Horizon. If you do not plan to use Horizon, you can simply remove it.

There’s an init container to optimize configuration and route loading. To share the application code between init container and app container, it uses an emptyDir.

There’s also affinity setting to tell Kubernetes to try it best to schedule the pods among different nodes as to avoid downtime.

CronJob

We use cronjob.yaml that leverage Kubernete’s CronJob to run php artisan schedule:run every minute. We feel like this is a more robust way of scheduling cron by fine-tuning activeDeadlineSeconds , backoffLimit and startingDeadlingSeconds to make sure our cron gets scheduled.

ConfigMap, Ingress & Service

Our ingress.yaml and service.yaml is pretty standard, we use CloudFlare DNS verification to obtain HTTPS certificates from Let’s Encrypt (It’s commented).

As for configmap, it’s recommended to use secret to store sensitive information like the database password.

Here’s a GitHub Repo in case you would like to clone it. Merge request is also appreciated!

mkhmylife/laravel-k8s
Contribute to mkhmylife/laravel-k8s development by creating an account on GitHub.github.com

Set up Gitlab CI

Now that we have our app and kubernetes config ready, we can go ahead to setup Gitlab CI to automate our deployment. (We assume you are using Kubernetes token authentication)

Setup CI/CD Environment Variables

Within Repo > Settings > CI / CD, we need to store our Kubernetes Cluster credentials into Gitlab CI’s environment variables.

Move the Dockerfile to your project’s root directory, then create a folder called k8 and store all the kubernetes yaml files inside. Create a file called .gitlab-ci.yml that contains the folllowing.

The CI/CD script contains two steps. First is to build our Laravel Application and push to Amazon ECR (you may configure it to another Docker Image Repo as you like). Then it moves on to deploy our Laravel Application image to our Kubernetes cluster.

In line 36 and 37, the $KUBE_URL and $KUBE_TOKEN are the two environment variables that we set up above.

In line 40, we ask kubectl to apply our k8s configuration files.

In line 41, it a hack to trigger redeployment of pods. Since we have set the imagePullPolicy to always, Kubernetes will automatically re-pull our docker image to the latest version. Combine with our deployment’s rolling update strategy, there’s should be no downtime in deployment updates.

In production, we actually use Kustomize to maintain deployment of multiple Git branches and environment. But we are looking to switch to Helm as it seems to be an easier and more popular deployment method.

img_6067

The joy of CI/CD. Push your code, wait for 4 minutes and it’s deployed automatically, without downtime.

And that’s it! the joy of CI/CD. Just push your code, wait for 4 minutes and it’s deployed automatically, without downtime.

laravel-k8s-master

djcelery redis做后台报错

 

报错内容：

 

pip list redis

>redis (3.2.0)

安装：pip install redis==2.10.6

后解决。

admin后台参数格式：

The arguments and keyword arguments must use double quotes.

So if you are specifying arguments, it should be like:

If you are specifying keyword arguments, it should be like this:

如何在django里上传csv文件并进行入库处理

运维平台导入数据这一功能实在是太重要了，我敢说在没有建自己的cmdb平台前，大多数公司管理服务器信息肯定是表格，用表格最麻烦的就是有点更新就得每个人发一份，这样大家信息才能统一，很不方便，终于有一天受不了了，搞了一个服务器信息管理平台，那面临的第一个问题不是说功能好或不不好，而是怎么才能把表里的数据导入到数据库中，所以你说重要不重要，当然如果你就喜欢自己手工录入（找虐的感觉），这个咱也不能说啥，各有所好嘛，那具体如何录的最快，这个不在我们今天的讨论范围，我只讨论如何自动导入。

提到导入，那一般有二个方法，一个是在前端上传完后存储在服务器上的某个目录里，然后读取文件进行分析处理。

另一种是上传文件后直接读取文件内容而不存储在服务器上，这二种方法都可以实现我们得目的，这篇主要是讨论的后面这种。

上传文件，首先我们建一个html文件，内容代码如下：

 

这些都是基本的Html,只要主要enctype=”multipart/form-data”这个参数就可以，其它无特别说明。

展示如图：

how-to-upload-and-process-the-csv

加入路由，

那接下来就是处理上传的文件并入库了，这个代码在views.py文件里，代码如下：

 

代码解释：

最开始判断如果是get请求直接渲染upload_csv.html文件，如果是post请求那么进行分析处理，首先是检查文件名是否是以csv结尾的，如果是就处理，不是就提示错误信息，再就是检查下上传文件的大小，其实这些检查也还好，如果是自己整理的表导入基本也不用做这些检查了，如果是有多人操作这些检查就有必要了，然后开始循环迭代文件行，内容是以逗号分隔，这里假设就是4列，如果你的表格列很多可以做修改，最后保存入库，如果有错误就记录错误信息到日志文件中。

至此我们需要的功能就完成了，虽然简单但非常实用。

Django ORM if you already know SQL

If you are migrating to Django from another MVC framework, chances are you already know SQL.

In this post, I will be illustrating how to use Django ORM by drawing analogies to equivalent SQL statements. Connecting a new topic to your existing knowledge will help you learn to use the ORM faster.

Let us consider a simple base model for a person with attributes name, age, and gender.

t7fs6oc

To implement the above entity, we would model it as a table in SQL.

The same table is modeled in Django as a class which inherits from the base Model class. The ORM creates the equivalent table under the hood.

 

The most used data types are:

SQL Django
INT IntegerField()
VARCHAR(n) CharField(max_length=n)
TEXT TextField()
FLOAT(n) FloatField()
DATE DateField()
TIME TimeField()
DATETIME DateTimeField()

The various queries we can use are:

SELECT Statement

Fetch all rows
SQL:

Django:

Fetch specific columns
SQL:

Django:

etch distinct rows
SQL:

Django:

Fetch specific number of rows
SQL:

Django:

LIMIT AND OFFSET keywords
SQL:

Django:

 

WHERE Clause

Filter by single column
SQL:

Django:

Filter by comparison operators
SQL:

Django:

BETWEEN Clause
SQL:

Django:

LIKE operator
SQL:

Django:

IN operator
SQL:

Django:

 

AND, OR and NOT Operators

SQL:

Django:

SQL:

Django:

SQL:

Django:

 

https://amitness.com/2018/10/django-orm-for-sql-users/

 

 

 

 

 

让我们来做个django小项目之二

上篇内容我们算是来了一个开场，创建了我们的项目，最后把数据库也建立完成了，这篇我们主要完成后台数据入库的部分，根据我们之前的思路，如果已经能实时获得每个站点性能信息了，如果能将每次获得信息插入到数据库中，这个就完成了我们的入库部分，这里说一下我们的数据库操作部分，要操作数据库，首先我们要跟数据库建立连接，然后进行常规的CRUD操作，操作完毕后再关闭数据库连接，这是一个基本流程，所以为了提高数据库的操作速度，我们可以把一些常用的操作封装成一个类，以下是我门数据库操作的类代码：

在这个类中我们先定义了__init__方法，指定了数据库连接，用户，端口，用户等信息，接下来是定义了三个方法，分别是建立数据库连接，获取数据内容、更新数据库及最后关闭数据库连接，正题逻辑就是这样，代码没特别的内容。

有了这个数据库类后，我们就可以用上我们之前的pycurl模块去获取每个站点的数据了，每次获取的数据插入到数据库中，因为可能非常多的站点，所以我们要考虑运行效率，所以我们这里使用并发去检查站点数据，采用ThreadPool线程池去实现，全部代码如下：

以上就是我们全部的代码内容，pycurl部分我们不在多解释，不理解的小伙伴可以参考之前的文章，Dummy就是多进程模块的克隆文件，唯一不同的是，多进程模块使用的是进程，而dummy则使用线程，pool.map类似python的map函数，需要二个参数第一个参数是一个函数名，第二个参数是个迭代对象，在这里迭代对象我们是从数据库中直接获取的，至于数据库的数据从哪里来的呢？当然是用户输入的，那下篇我们实现下前端输入页面，让用户自己可以增加要监控的站点，这篇就到这里，喜欢的小伙伴请帮忙转发哟。