Download online registration system and more Assignments Research Methodology in PDF only on Docsity! Student: Jose García Pérez Professor: Hongying Gu PFC -‐ Final thesis Z h e j i a n g U n i v e r s i t y -‐ 浙江大学(中国) F a c u l t a t d ' I n f o r m à t i c a d e B a r c e l o n a T e c h n i c a l U n i v e r s i t y o f C a t a l o n i a Zhejiang University Online registration system Ch ap te r: In tr od uc tio n 2 Table of contents Introduction ....................................................................................................................................................................................... 3 Current problem .......................................................................................................................................................................... 3 Proposal solution ............................................................................................................................................................................. 5 Description ..................................................................................................................................................................................... 5 General scheme ................................................................................................................................................................................. 7 Registration process .................................................................................................................................................................. 7 Scheme .................................................................................................................................................................................................. 8 Design .................................................................................................................................................................................................... 9 RMI .................................................................................................................................................................................................... 9 Summary .................................................................................................................................................................................... 9 Decisions ................................................................................................................................................................................. 11 Security .................................................................................................................................................................................... 12 Implementation details .................................................................................................................................................... 13 Databases .................................................................................................................................................................................... 20 Summary ................................................................................................................................................................................. 20 Table details .......................................................................................................................................................................... 21 Web ................................................................................................................................................................................................ 23 Summary ................................................................................................................................................................................. 23 Implementation ................................................................................................................................................................... 25 Security ......................................................................................................................................................................................... 27 Summary ................................................................................................................................................................................. 27 Password policy ................................................................................................................................................................... 28 Parameter protection ....................................................................................................................................................... 29 Private web zone protection .......................................................................................................................................... 29 Load balancing .......................................................................................................................................................................... 30 Summary ................................................................................................................................................................................. 30 Implementation ................................................................................................................................................................... 30 Load tests ..................................................................................................................................................................................... 32 Used software ....................................................................................................................................................................... 32 Used hardware ..................................................................................................................................................................... 32 Test design ............................................................................................................................................................................. 32 Results ...................................................................................................................................................................................... 34 Achieved objectives ...................................................................................................................................................................... 37 Ch ap te r: P ro po sa l s ol ut io n 5 Proposal solution Description As described in the previous section, there are three problems to be solved separately. However, they are completely interrelated. For the problem of central servers, the solution is not simply to replicate them, since the problem cannot be simply the ability to process, but also the synchronization between them, the input data synchronization of the databases or the central server bandwidth. The solution is to make each faculty manage their registrations, which will free the loading process to central servers while the bandwidth is distributed to each faculty. There are some difficulties that must be taken into account: a) Security is an essential feature which has much more importance than others features. The system must ensure that the one who is performing the registration process is the one who is supposed to be. b) After the registration, the server will have the ability to connect with the central servers and insert information into their databases. This server-‐server connection will involve, apart from adding a security layer, a system of synchronization between servers. Again, all of this to avoid the collapse the central server connections, which is the main goal of the project. c) About the online registration: a. Due to the registration is done through the Internet, the system should take into account the high number of connections in a given time, which could become at most all the students that are supposed to be registered that day. Therefore, the implemented system cannot allow the collapse of the faculty servers. b. On the other hand, the implemented system must respect the pre-‐ established registration order (student priority). c. At any given time the load of a server that manages user access can be very high and therefore, the system implementation must divide the bandwidth and data load between one or more servers. Ch ap te r: P ro po sa l s ol ut io n 6 d. Apart from implementing a system of queues to respect the priority of a student, the system implementation cannot allow to skip the queue directly accessing the server that manages the implementation of the registration of the faculty. In addition, this checksum must spend the minimum possible resources to avoid the maximum collapse of the system. e. The registration database of the faculty must be updated each time a student completes its registration, decreasing the number of places available for each subject in each group that the student chose. The students will be able to check in real time if the groups of the subjects they want to attend are still available. Since the number of students can be very large at any given time (all of them could be querying at the same time). The real time consultation will constantly throws many queries to the database. Due to this feature, it could happen that the database become totally collapsed of connection, which would leave the system totally unusable. To avoid this, the system implementation must minimize the number of simultaneous connections to databases. Ch ap te r: G en er al sc he m e 7 General scheme Registration process 1. A student accesses the URL of the Web application. 2. This URL takes to a load balancing system that handles the request and redirects it to one of its servers. 3. The student must authenticate him/herself. 4. The student accesses the subject and group list available for him/her. 5. Once the enrollment system is turned on, the student will access a website and his/her request will be inserted in the queue of that server respecting the registration priority order previously given by his/her faculty. 6. This dynamic website will inform the student of the latest student which started the enrollment process and how many students are actually in the registration server. That will help the student to calculate the approximate time they must wait to finish the process. 7. Once the faculty server checks that this student has the highest priority of all the queues (different servers, different queues), he/he will be allowed to freely decide which subjects and groups want to enroll (only if they are still available). 8. The student will complete the registration once he/she has correctly chosen the subjects and groups to register. Now this data will be stored in the database of the faculty. This is the end of the process. Ch ap te r: D es ig n 1 0 (FrontEndListClient) as well as the common interface between two servers (FrontEndList). The client it’s only called when the front-‐end server is booted. -‐ Faculty server as client (StudentQueueFacultyServerClient) with threads, in order to be compatible with the service FrontEndList described above. The client performs a polling to all front-‐end servers checking who is the student with more priority and also, it generates 128 bytes tokens once it have free positions. -‐ Central server as client (InsertServiceClient), which performs a polling to the faculties servers. Ch ap te r: D es ig n 1 1 Decisions The following decisions were taken for the implementation of these classes: -‐ The server and the implementation of the service StudentQueue is integrated in the init() function of the tomcat server. Considering the front end is both a client and a server, perform the implementation in this way it’s a good decision. -‐ To make an efficient connection between the tomcat and the registration server, and because in the faculty server the RMI is not integrated on the tomcat, there is a table in the database named registrations to manage the authorized students. Moreover, considering that the server should handle the power of a limited number of connections, the fact of using a table in the database does not subtract efficiency and nor pose threat to the integrity of the system. Finally, to control the timeout of a student who is waiting in a front-‐end server, the front-‐end server must notify the registration server that the student is no longer there. Thus, the database avoids the problem of using a servlet as an RMI client. -‐ A client can be connected simultaneously from 2 different computers, which may result in a client over more than a front-‐end. To ensure that the student is advised for all servers where is connected and to not spend more faculty slots of the registration server, when the registration server detects that the student has been already advised, it will send the same token as in the database. Ch ap te r: D es ig n 1 2 Security -‐ Creation of SSL server certificates for the central server. -‐ Integration of SSL with mutual authentication, making use of the certificates created by tomcat in order not to double the number of certificates on the servers. -‐ To prevent a student to skip the order of registration, the system generates a unique 128 bytes token for every single student and priority. Ch ap te r: D es ig n 1 5 1) Locate StudentQueue. 2) Add to the query server list. If list size = 1, run thread while(1) Ch ap te r: D es ig n 1 6 StudentQueue The front end server is the responsible of publishing the StudentQueue service, which is the server that allows the registration server to check who are the students with more priority in each front end server as well as send a token to the n students with more priority among all front end servers, so this one is advised by ajax. Sequence diagram Ch ap te r: D es ig n 1 7 First client (first function) It is a call from the client (registration server) where each front-‐end server is asked for its first student in the queue. At the same time, it sends by parameter who is the last student allowed. Because the server is in fact the web server, ajax will update the waiting page of the student. As a result of the function, the student with more priority in this front-‐end server is got in order to be compared with the others so it gets the one with more priority in all the front-‐end servers. Registration Allowance It is a call from the client (registration server), which informs the front end that the student with the turn passed by parameter is now allowed to register. At the same time, a token is passed so tomcat using ajax, generates a button where the HTTPS petition includes it. The result of the function is a true if the student with that priority is no longer in the queue (exception) and false if everything worked as expected. StudentQueueRegistrationServerClient The implementation of the client is a thread that is periodically executed and does the following: 1) For each front-‐end server in its list, execute the first function. 2) Compare the obtained results and get the highest priority number. 3) Query the authorizations database and check the count of authorized students. a. If the number is less than the max i. Check if the token for this student has already been created. If not, create a new one ii. Call the RegistrationAllowance function with parameters priority and student token. 1. If everything is ok, insert token into the authorization database. Ch ap te r: D es ig n 2 0 Databases Summary To deploy the database server MySQL was chosen for being an Open Source product prestigious and powerful documentation and support. There are three databases: Faculty Core database of the faculty with the main information of students and subject availability for each of the students. Registration The database used by the registration server. Its function is to take control of the access of the students and their tokens as well as save the registration information for later be able to pass it to the central server. Central The database of the central server. It saves the information of the faculties that has to be connected and also the registrations of all the students of all faculties. The databases have been designed so that queries that have to do the different systems could be resolved by a single table, which make them more efficient. This is important due to, with all the features of the system, without this design, the database could result in a bottleneck. Also because of this, the queries launched by each front-‐end server to show the availability of places for each student, has been resolved by a single thread. This thread is executed every five seconds and updates the information about free places of all the groups on the front-‐end servers. After that, the front-‐end servers have the information ready to be shown to each student according to the subjects that can be enrolled by him/her. For security reasons, for each server and database, different users and passwords have been defined as well as read/write permissions for each of them. SHA1 encryption has been used to store the passwords. Ch ap te r: D es ig n 2 1 Table details Following are shown details for each database described before. Faculty Contains the following tables: subjects: Information about the subjects available in the faculty Field Type Null Key Default subject_id varchar(20) NO PRI subject_name varchar(128) NO credit_num double NO students: Information about the students of the faculty. Field Type Null Key name varchar(45) NO PRI password varchar(256) NO address varchar(128) NO phonenr varchar(20) NO email varchar(45) NO bank_account varchar(45) NO groups: Information about free places of the groups of the subjects Field Type Null Key subject_id varchar(20) NO PRI group int(11) NO PRI free_spots int(11) NO priorities: Contains the priority number for each student. Field Type Null Key Default priority int(10) unsigned NO PRI name varchar(45) NO MUL password varchar(256) NO registered tinyint(4) NO 0 Ch ap te r: D es ig n 2 2 student_subject: Contains the selective subjects by each student. Field Type Null Key Default student_name varchar(45) NO PRI subject_id varchar(20) NO PRI Registration Contains the following tables: registrations: information about realized registrations. Field Type Null Key Default reg_id int(11) NO PRI student_name varchar(45) NO PRI subject_id varchar(20) NO PRI group int(11) NO credit_nr double NO price double NO frontends: information about frontends. Field Type Null Key Default ip varchar(20) NO PRI active tinyint(4) NO 0 authorizations: information for controlling the access of the students to the server. Field Type Null Key Default position int(11) NO PRI token char(255) YES null connected tinyint(4) NO 0 Central faculty: Contains information about all the faculties of the university. Field Type Null Key Default name varchar(100) YES ip varchar(20) YES port int(11) YES Start_date timestamp NO CURRENT_TIMES TAMP Ch ap te r: D es ig n 2 5 Implementation The system is based in the following two tools: -‐ Web Application Server: Tomcat 6.0 -‐ Struts Framework 1.0 Because the usage of the resources in a front-‐end can be very high (the total number of the students divided by the number of the working Front End servers in that precise moment) , the programming of this application has been made minimizing the memory usage per session, minimizing the connections to databases and agility in the source code in order to reduce response time. As regards security, the HTTPS protocol has been used in both Web servers allowing safe navigation and protects the service of phishing attacks. Prototype has been used for the use of AJAX . It allows parts of a web updates every defined time. This framework is called using JavaScript. The layout has been done using CSS so the presentation layer is separated from the HTML. Due to this, the style can be easily changed without any modification of the HTML/JSP. Following sections show the implementation for each operation on the website. Ch ap te r: D es ig n 2 6 FrontEnd Context Inicialized function This function is called just when Tomcat starts. During this operation, threads are called, an instance of the RMI service is created and it does a rebind of it. On the other hand, the variables defined on the context.xml file are loaded on the constants class. Update Availables The part of the system responsible for showing the free spots on the selective subject list. It uses HTTPS with AJAX for communication between the server and the client and RMI for the communication to/from the central server. This operation is realized inside the action controller and it periodically updates using AJAX. The user will not be able to make any further step until the operation UpdatePreparedStudents is executed. On the other hand, during the execution of this part, the last authorized student priority will be shown by the use of the operation "update last registered". CaptureSubjects This operation is called inside the action controlled named CaptureSubjects.java It generates an output HTML file that shows the subjects and selected groups and also the personal details of the student. In this point, he or she is able to edit these details. If there was a problem with access to this part, the system will show a screen recommending that the user be directed back to the beginning of the system. This measure is taken to ensure that only legitimate users access the application. Ch ap te r: D es ig n 2 7 If there was no problem at all, at this moment the user has the reserved spots reserved so nobody else can take them. Do Registration This operation is realized inside the action controller "DoRegistration.java". The result is a generation of an html file that shows the subjects and selected groups with the number of credits, the complete name and the price of each subject and the total. First Returns the priority of the first person that is on the FrontEnd queue, if there is nobody it returns a 0. RegistrationPermission Is an RMI call that has instantiated service within the context of Tomcat. Its two steps are: • Update the Tomcat global variable "GrantedStudents". It contains a list of users and tokens that are able to access to the registration server. • Delete the student from the queue. • Return true or false depending if any internal problem happened. Security Summary The role of the security package is to determine the sensitive points to external attacks as long as defining how to prevent them. The following are the main tasks of the security package: • Define a password policy. • Encryption of sensitive data. • Protection of the parameters received by the system. • User Credentials and server. • Private web zone protection. Ch ap te r: D es ig n 3 0 Load balancing Summary Load balancer is a technology that serves to make a balanced load between several network services such as web servers or email. This technology applies to the layer 4 (switching) of the Linux kernel. This allows TCP and UDP connections to balance the load among several real servers. The main advantage of the load balancer is the ability to divide a multiple connection service to various servers that treats all requests equally, but where the client does not connect directly. This prevents a potential collapse problem when trying to connect too many clients to one server. The operation of the load balancer is easy. Instead of clients to directly connect to one server that provides the same service, connect to a load balancer server that is responsible for distributing the burden equitably among all requests to all the available real servers. Load balancer uses Keepalived technology. This enables constant communication between real servers and the load balancer. That is, it makes it possible to add a new real server and make the load balancer to detect on the fly the change in the system. So, once a new server is added to the network, the load balancer automatically adds it to its real server distribution list. Also deals with knowing when a server its down so the load balancer doesn't send more petitions. Implementation Due to resource limitations in the system, is not possible to apply load balancer technology in TCP layer 4. As there is only a single server, this is, front-‐end servers are not in different computers (different IP's) but virtually deployed on the same server (different ports). The system design can be perfectly distributed among different machines, which is why the load balancer configuration files are included even for the tests that are performed those are not not going to be used. Ch ap te r: D es ig n 3 1 For the performed tests, the system has a pseudo load balancer. This is a website that merely checks in a database which front end servers are available and distribute the requests between these equally (1 mod Number of available servers). When a server crashes or is added, the change is automatically entered into the database so that the next request will be distributed among the new number or available servers. Obviously the effectiveness of this system is less than a real load balancer. Still, the function to perform for this project is exactly the same. Ch ap te r: D es ig n 3 2 Load tests Used software It is mandatory to test the reliability of the system since its one of the goals in this project. To prove it, its necessary to realize several load tests. To perform those tests, it's been decided to use JMeter due to the Java implementation of the whole system. JMeter allows evaluating the system performance from the client side (response time and error percentage). Anyway, since the goal is not only to check the performance from the client side but also from the server side, some shell scripts has been implemented to measure the system resources. Those shell scripts are an easy a reliable way to obtain the values of the parameters such as memory and CPU consumption. Used hardware As there is a limitation of physical resources, the whole system has been deployed and developed in a single computer. The technical specs are shown in the following table: Model Macbook Pro 5.5 OS MAC OS X Snow Leopard 10.6.5 Java version 1.6 RAM Memory 4 GB 1067 MHz DDR3 CPU Intel Core 2 Duo 2,53 GHz Number of deployed front-‐ends 3 Test design The goal is to measure the following variables based on the number of users concurrently online: • Client side o Response time o Error percentage • Server side (resource consumption) o CPU o Memory Ch ap te r: D es ig n 3 5 The most notable of these graphs is that the error percentage of the registration server is always 0. This is the server that let pass the requests of the balanced servers asking for them when it's not overloaded. Since the goal is precisely that this server is never down, this graph proves that the goal is achieved. Ch ap te r: D es ig n 3 6 Here are the results obtained with the shell script: Ch ap te r: A ch ie ve d ob je ct iv es 3 7 Achieved objectives 1. A solid, easily adaptable and with minimal chance of failure registration system that prevents saturation of the central server has been built. 2. The system also solves the initial problem of having to attend faculty to enroll subjects and it perfectly respects the priority registration system. 3. Also the system ensures that there is no possibility of loosing the spot once the reservation is done. 4. A friendly user interface that shows and lively updates the number of free spots. Possible improvements The initial intention was to deploy an entire system on several servers; this would have become an improvement in the results of the load tests and better treatment of the whole system. In the case of the web, it would be great to improve the response time on the front-‐ end servers. This is probably just a matter about adapting the tomcat startup settings and services to the machine in use. About the RMI, a possible improvement could be including the initialization of the service when the tomcat server starts, exactly the same way that it was done on the front-‐end server, preventing the boot of 2 different processes. In the case of the database could be trying to create a local database in each front-‐ end with static data to speed up some queries. About the system security, the use of certificates provided by authorities would be not only a great improvement but also necessary for a real deploy.