Preface
In order to solve the various problems brought by the traditional web development model, we have made many attempts, but due to the physical gap between the front and back ends, the solutions we tried are similar. Learning from the pain, today we rethink the definition of "front and back end" and introduce NodeJS, which is familiar to front-end students, trying to explore a brand new front-end separation mode.
With the rise of different terminals (Pad/Mobile/PC), developers have increasingly high requirements. The responsiveness of the pure browser side can no longer meet the high requirements of user experience. We often need to develop customized versions for different terminals. In order to improve development efficiency, the need for separation of front and back ends is increasingly valued. The back end is responsible for business/data interfaces, the front end is responsible for display/interaction logic, and we can customize and develop multiple versions of the same data interface.
This topic has been discussed a lot recently, and some Alibaba BUs are also making some attempts. After a long discussion, our team decided to explore a set of front-end and back-end separation scheme based on NodeJS. There were some changing understandings and thoughts in the process. I recorded it here. I also hope that the students I saw participated in the discussion and helped us improve it.
1. What is front-end separation?
During the initial discussion within the group, I found that everyone has a different understanding of front-end separation. In order to ensure that they can discuss on the same channel, we will first reach an agreement on what "front-back-end separation" is.
An example of front-end separation that everyone agrees with is SPA (Single-page application). All the presentation data used is provided by the backend through the asynchronous interface (AJAX/JSONP), and the front-end only displays it.
In a sense, SPA does achieve front-end separation, but there are two problems with this method:
Among WEB services, SPA accounts for a very small proportion. In many scenarios, there is also a synchronous/synchronous + asynchronous hybrid mode, and SPA cannot be used as a general solution.
In the current SPA development model, the interface is usually provided according to the presentation logic. Sometimes, in order to improve efficiency, the backend will help us handle some presentation logic, which means that the backend is still involved in the work of the View layer, not the real separation of front and backends.
SPA-style front-end separation is distinguished from the physical layer (think that as long as it is the client, the front-end, and the back-end is the server). This classification can no longer meet our needs for front-end separation. We believe that only by dividing responsibilities can we meet our current usage scenarios:
Front-end: Responsible for View and Controller layers.
Backend: Only responsible for Model layer, business processing/data, etc.
Why do this division of responsibilities will be discussed later.
2. Why should we separate the front and back ends?
Regarding this issue, Yu Bo's article explains it very comprehensively in the evolution of the Web R&D model. Let's briefly review it:
2.1 Applicable scenarios for existing development models
The development models mentioned by Yu Bo each have their own applicable scenarios, and none of them completely replace the other one.
For example, for MVC, which is mainly based on the backend, it is very efficient to perform some synchronous display business, but when encountering a synchronous and asynchronous page, it will be more troublesome to communicate with the backend development.
Ajax is the main SPA development model, which is more suitable for developing APP-type scenarios, but it is only suitable for making APPs, because SEO and other problems are difficult to solve. For many types of systems, this development method is also too heavy.
2.2 The front and back end responsibilities are unclear
In systems with complex business logic, we are most afraid of maintaining the code mixed with the front and back ends. Because there is no constraint, the code of other layers of MVC may appear in each layer. Over time, there is no maintenance at all.
Although the separation of front and back ends cannot completely solve this problem, it can be greatly alleviated. Because it is guaranteed from a physical level that you cannot do this.
2.3 Development efficiency issues
Taobao’s web is basically based on the MVC framework webx, and the architecture determines that the front-end can only rely on the back-end.
So our development model is still that the front-end writes static demos and the back-end translates them into VM templates. I won’t talk about the problem with this model, and I have been criticized for a long time.
It is also painful to directly develop based on the back-end environment, and it is troublesome to configure, install and use. In order to solve this problem, we invented various tools, such as VMarket, but the front-end still has to write VMs, and rely on back-end data, so the efficiency is still not high.
In addition, the backend cannot get rid of the strong focus on display and thus focus on the development of the business logic layer.
2.4 Limitations on front-end performance
If performance optimization is only done on the front end, we often need backend cooperation to create sparks. However, due to the limitations of the backend framework, it is difficult for us to use Comet, Bigpipe and other technical solutions to optimize performance.
In order to solve some of the problems mentioned above, we have made many attempts and developed various tools, but there has never been much improvement, mainly because we can only use the small piece of space divided by us in the backend. Only by truly separating the front and back ends can we completely solve the above problems.
3. How to separate the front and back ends?
How to separate the front and back ends? In fact, there is an answer in the first section:
Front-end: Responsible for View and Controller layers.
Backend: Responsible for Model layer, business processing/data, etc.
Just imagine, if the front-end masters the Controller, we can do url design, we can decide whether to synchronize rendering on the server according to the scene, or output json data based on the view layer data, and we can also easily do Bigpipe, Comet, Socket, etc. according to the needs of the presentation layer. It is entirely the use method determined by the requirements.
3.1 "full stack" development based on NodeJS
If you want to implement the layering of the above figure, you will inevitably need a web service to help us realize what we do before and after the backend, so there is the title "full-stack development based on NodeJS"
This picture looks simple and easy to understand, but hasn't tried it and there will be many questions.
In SPA mode, the backend has provided the required data interface, and the view frontend can already be controlled. Why add the NodeJS layer?
How about adding one more layer?
Adding one more layer will increase the workload of the front-end?
Adding one more layer will lead to another layer of risk. How to break it?
NodeJS can do anything, why do you still need JAVA?
It is not easy to explain these issues clearly. Let me talk about my understanding process below.
3.2 Why add a layer of NodeJS?
At this stage, we mainly develop the backend MVC model. This model seriously hinders the efficiency of front-end development and prevents the back-end from focusing on business development.
The solution is to enable the front-end to control the Controller layer, but it is difficult to do it under the existing technology system, because it is impossible to let all front-ends learn Java, install the back-end development environment, and write VMs.
NodeJS can solve this problem very well. We can do what the development helped us do before, and everything seems so natural.
3.3 Performance issues
Layering involves communication between each layer, and there will definitely be certain performance losses. However, reasonable stratification can make responsibilities clear and collaborative, which will greatly improve development efficiency. The losses caused by stratification will definitely be compensated for in other aspects.
In addition, once we decide to tie up, we can minimize losses by optimizing communication methods and communication protocols.
For example:
After the Taobao Baby Details Page is static, there is still a lot of information that needs to be obtained in real time, such as logistics, promotions, etc. Because this information is in different business systems, the front-end needs to send 5 or 6 asynchronous requests to backfill these contents.
With NodeJS, the front-end can proxy these 5 asynchronous requests in NodeJS, and can easily do Bigpipe. This optimization can improve the entire rendering efficiency a lot.
You may think it's okay to send 5 or 6 asynchronous requests on the PC, but on the wireless side, it's very expensive to establish an HTTP request on the customer's mobile phone. With this optimization, the performance is several times improved.
Taobao details are based on NodeJS optimization. We are in progress. After it is launched, I will share the optimization process.
3.4 Has the front-end workload increased?
Compared to just cutting pages/demos, it must be a little more, but there are linkages and communications in the current mode. This process takes a lot of time, is prone to bugs, and is difficult to maintain.
Therefore, although the workload will increase a little, the overall development efficiency will be greatly improved.
In addition, testing costs can be saved a lot. The interfaces developed in the past are all aimed at the presentation layer, and it is difficult to write test cases. If the front and back end separation is done, even the test can be separated. One group of people specializes in testing the interface and the other group of people focuses on testing the UI (this part of the work can even be replaced by tools).
3.5 How to control the risks brought by adding the Node layer?
With the large-scale use of Node, students from the system/operation and maintenance/security department will definitely join the infrastructure construction. They will help us improve possible problems in each link and ensure the stability of the department.
3.6 Node can do anything, why do you still need JAVA?
Our original intention is to separate the front and back ends. If we consider this issue, it will be a bit contrary to our original intention. Even if we use Node to replace Java, we cannot guarantee that we will not encounter any problems we encounter today, such as unclear responsibilities. Our goal is to develop in a layered manner, professional people, and focus on doing professional things. The infrastructure based on JAVA is already very powerful and stable, and is more suitable for doing what is now architecture.
4. Taobao's Node-based front-end separation
The above picture is what I understand on Taobao’s front-end and back-end separation and layering based on Node, as well as the scope of responsibilities of Node. A brief explanation:
The top end is the server, which is what we often call the backend. For us, the backend is a collection of interfaces, and the server provides various interfaces for us to use. Because there is a Node layer, there is no need to limit what form of service it is. For backend development, they only use interfaces that care about business code.
The server is under the Node application.
There is a layer of Model Proxy in the Node application that communicates with the server. This layer is mainly used to smooth out the way we call different interfaces and encapsulate some models needed by the view layer.
The Node layer can also easily achieve the original vmcommon, tms (citing Taobao content management system) and other requirements.
What framework to use for the Node layer is up to the developer to decide. However, it is recommended to use a combination of express + xTemplate, which can be used for the front and back ends.
Everyone decides how to use Node, but what is exciting is that we can finally use Node to easily implement the output method we want: JSON/JSONP/RESTful/HTML/BigPipe/Comet/Socket/Synchronous and asynchronous. It can be done whatever you want, and it is decided entirely based on your scenario.
The browser layer has not changed in our architecture, and we do not want to change your previous understanding of the development in the browser because of the introduction of Node.
Introducing Node is just to hand over the front-end control part that should be controlled by the front-end.
We have two projects in development in this model. Although they have not yet been launched, we have already tasted the sweetness in terms of development efficiency and performance optimization.
5. What else do we need to do?
Integrate Node's development process into Taobao's existing SCM process.
Infrastructure construction, such as session, logger and other general modules.
Best Development Practices
Online success stories
Everyone’s understanding of the concept of separation of Node front and back ends
Safety
performance
…
There will not be too much technology that needs innovation and research, and there has been a lot of ready-made accumulation. In fact, the key is the opening up of some processes and the accumulation of general solutions. I believe that with more project practices, this area will gradually become a stable process.
6. "Midway Island"
Although the "full-stack development based on NodeJS" model is exciting, there is still a lot to go to transform the full-stack development based on Node into something that everyone can accept. Our ongoing "Midway" project is to solve this problem. Although we are not long before, we are getting closer and closer to our goal! !