# Using Apache Jmeter to test the ScholarX backend **URL:** \ **Category:** ScholarX\ **Created:** [June 24, 2021, 8:49am UTC](https://sef.discourse.group/t/using-apache-jmeter-to-test-the-scholarx-backend/308 "2021-06-24T08:49:30Z")\ **Posts on this page:** 16\ **Page:** 1
**Author:** ![piumal1999](https://yyz2.discourse-cdn.com/free1/user_avatar/sef.discourse.group/piumal1999/32/274_2.png) [@piumal1999](https://sef.discourse.group/u/piumal1999)\ **Post date:** [June 24, 2021, 8:49am UTC](https://sef.discourse.group/t/using-apache-jmeter-to-test-the-scholarx-backend/308/1 "2021-06-24T08:49:30Z")
Apache Jmeter is Java application designed to load test functional behavior and measure performance. In a previous discussion, @YohanAvishke suggested to use it for testing performance of the ScholarX backend. [https://jmeter.apache.org/](https://jmeter.apache.org/) Basically we can use this tool to create scripts to simulate the user requests. Ex: Simulate the backend part of mentor flow parallelly with 100 users There’s also a non-GUI version of this tool which can be added as an maven plugin. So we can include this in the tests also. cc: [@EngTeam](https://sef.discourse.group/groups/engteam) ---
**Author:** ![YohanAvishke](https://yyz2.discourse-cdn.com/free1/user_avatar/sef.discourse.group/yohanavishke/32/106_2.png) [@YohanAvishke](https://sef.discourse.group/u/YohanAvishke)\ **Post date:** [June 24, 2021, 2:47pm UTC](https://sef.discourse.group/t/using-apache-jmeter-to-test-the-scholarx-backend/308/2 "2021-06-24T14:47:58Z")
Thank you soo much @piumal1999 for working on the task. For the testing let’s use two thread groups, 1. For mentors (about 10 users) 2. For mentees (about 10 users for each mentor, all together 100) Let’s run test plans for each group synchronously. And by the end of the run the app will be tested against 110 users. Can you please post your test plans for each of the groups? ---
**Author:** ![piumal1999](https://yyz2.discourse-cdn.com/free1/user_avatar/sef.discourse.group/piumal1999/32/274_2.png) [@piumal1999](https://sef.discourse.group/u/piumal1999)\ **Post date:** [June 25, 2021, 7:26pm UTC](https://sef.discourse.group/t/using-apache-jmeter-to-test-the-scholarx-backend/308/3 "2021-06-25T19:26:37Z")
> [@YohanAvishke](#): > > For the testing let’s use two thread groups, > > 1. For mentors (about 10 users) > 2. For mentees (about 10 users for each mentor, all together 100) > > Let’s run test plans for each group synchronously. And by the end of the run the app will be tested against 110 users. Yeah, this makes sense. Btw, how do we manage the program state changes with this. I just tried using the tool with public endpoints. It works fine with the public APIs (which doesn’t require authentication) ![image](https://global.discourse-cdn.com/free1/uploads/sef/original/1X/01f77a82acae573c2a471894a154f88d99d89e21.jpeg) I’m still finding a way to test the other endpoints without the user registration part. ---
**Author:** ![YohanAvishke](https://yyz2.discourse-cdn.com/free1/user_avatar/sef.discourse.group/yohanavishke/32/106_2.png) [@YohanAvishke](https://sef.discourse.group/u/YohanAvishke)\ **Post date:** [June 26, 2021, 4:25am UTC](https://sef.discourse.group/t/using-apache-jmeter-to-test-the-scholarx-backend/308/4 "2021-06-26T04:25:29Z")
> [@piumal1999](#): > > how do we manage the program state changes with this. We can change the states manually. Only `mentee_application` and `mentor_application` are the states we should run the tests for. Also since the 2 thread groups will run synchronously after running the first thread group for mentors, we can change the state to `mentor_application` and run the second thread group. > [@piumal1999](#): > > I’m still finding a way to test the other endpoints without the user registration part. Can you have a-look into the link. Even though in the blog they use different SSO provider the mechanism should be the same. > **[Using Apache JMeter to run load test on a Web applications protected by...](https://www.dbi-services.com/blog/using-apache-jmeter-run-load-test-web-applications-protected-microsoft-advanced-directory-federation-services/)** > > One of my last mission was to configure Apache JMeter for performance and load tests on a Web Application. The access to this Web Application requires authentication provided by a Microsoft Advanced Directory Federation Services single Sign On... I also linked the part covering the login scenario, [https://jmeter.apache.org/usermanual/build-web-test-plan.html#logging\_in](https://jmeter.apache.org/usermanual/build-web-test-plan.html#logging_in) ---
**Author:** ![piumal1999](https://yyz2.discourse-cdn.com/free1/user_avatar/sef.discourse.group/piumal1999/32/274_2.png) [@piumal1999](https://sef.discourse.group/u/piumal1999)\ **Post date:** [June 30, 2021, 8:18am UTC](https://sef.discourse.group/t/using-apache-jmeter-to-test-the-scholarx-backend/308/5 "2021-06-30T08:18:48Z")
> [@YohanAvishke](#): > > Can you have a-look into the link. Even though in the blog they use different SSO provider the mechanism should be the same. I set up the configuration according to this but after setting the proxy server, Google chrome can’t access internet. I’ll find another method to do so. Refered this too: [![](https://global.discourse-cdn.com/free1/uploads/sef/original/1X/84a466d02462261923db93ea4d327c9053d88831.jpeg "JMeter Browser Recording Tutorial") ](https://www.youtube.com/watch?v=zYAiBwJK1v8) ---
**Author:** ![piumal1999](https://yyz2.discourse-cdn.com/free1/user_avatar/sef.discourse.group/piumal1999/32/274_2.png) [@piumal1999](https://sef.discourse.group/u/piumal1999)\ **Post date:** [July 3, 2021, 9:29am UTC](https://sef.discourse.group/t/using-apache-jmeter-to-test-the-scholarx-backend/308/6 "2021-07-03T09:29:53Z")
It worked with Firefox. So i was able to get the login records to mock the authentication process. After running those records manually, still it throws the 403 errors when accessing the endpoints. In most of the tutorials, it says to update the SAMLRequest, RelayState and SAMLResponse with a regex. But in our scenario, those details doesn’t show up in the login requests ---
**Author:** ![piumal1999](https://yyz2.discourse-cdn.com/free1/user_avatar/sef.discourse.group/piumal1999/32/274_2.png) [@piumal1999](https://sef.discourse.group/u/piumal1999)\ **Post date:** [July 7, 2021, 10:29am UTC](https://sef.discourse.group/t/using-apache-jmeter-to-test-the-scholarx-backend/308/7 "2021-07-07T10:29:51Z")
Since the authentication part was not working in Jmeter, I ran a test with only using the public endpoints. I added the 3 public endpoints to the test. - Get all programs - Get program by id - Get mentors of a program ![image](https://global.discourse-cdn.com/free1/uploads/sef/original/1X/18c857fd9c04c6573f33151924900ec78d632055.png) I used 100 thread, so each endpoint runs 100 times at once. But only 86 threads completed well this time. Maybe heroku is blocking the remaining requests. ![image](https://global.discourse-cdn.com/free1/uploads/sef/original/1X/4dd6c8255eeaaf692ca6ea533dc41695483713f8.jpeg) So 259 out of 300 requests succeeded. The rest was not running. After stopping it, I ran the 100 threads again but the cleardb database reached the query limit after few requests (nearly 80). So I have to wait until the query limit resets for another attempt. ![image](https://global.discourse-cdn.com/free1/uploads/sef/original/1X/650761973c9a7b5878b318a4ac3a095ce2c75238.png) cc: @YohanAvishke [@EngTeam](https://sef.discourse.group/groups/engteam) ---
**Author:** ![piumal1999](https://yyz2.discourse-cdn.com/free1/user_avatar/sef.discourse.group/piumal1999/32/274_2.png) [@piumal1999](https://sef.discourse.group/u/piumal1999)\ **Post date:** [July 7, 2021, 2:37pm UTC](https://sef.discourse.group/t/using-apache-jmeter-to-test-the-scholarx-backend/308/8 "2021-07-07T14:37:41Z")
This is the result I got after 3 hours. ![image](https://global.discourse-cdn.com/free1/uploads/sef/original/1X/2e5e1a58b8b558330d6ca6973549cb88cb361572.jpeg) ---
**Author:** ![jaye](https://yyz2.discourse-cdn.com/free1/user_avatar/sef.discourse.group/jaye/32/810_2.png) [@jaye](https://sef.discourse.group/u/jaye)\ **Post date:** [July 9, 2021, 7:12am UTC](https://sef.discourse.group/t/using-apache-jmeter-to-test-the-scholarx-backend/308/9 "2021-07-09T07:12:15Z")
This is nice @piumal1999 ! What’s the conclusion of your result as you see? Is the current deployment feasible of running the pilot? ---
**Author:** ![piumal1999](https://yyz2.discourse-cdn.com/free1/user_avatar/sef.discourse.group/piumal1999/32/274_2.png) [@piumal1999](https://sef.discourse.group/u/piumal1999)\ **Post date:** [July 9, 2021, 5:27pm UTC](https://sef.discourse.group/t/using-apache-jmeter-to-test-the-scholarx-backend/308/10 "2021-07-09T17:27:46Z")
As I see the server is enough for the pilot. The problem we have is with the database. Since clearDB has a query limit per hour, we’ll have to use either PostgreSQL or a database in another server. Btw, I ran the tests again had the same result. So there shouldn’t be anything wrong with the server. First 100 threads (0% error rate): ![Screenshot 2021-07-09 at 21.06.13](https://global.discourse-cdn.com/free1/uploads/sef/original/1X/802d5b8ad352186919dedc88c7060f9e886a9167.jpeg) Another 100 threads (nearly 50% error rate due to the cleardb problem): ![Screenshot 2021-07-09 at 21.09.19](https://global.discourse-cdn.com/free1/uploads/sef/original/1X/2a7363b7f7228e67dfaf055044b2dbbf1b9735d7.jpeg) My conclusion: Need to use another way to keep the database WDYT? @jaye @YohanAvishke @Gravewalker @anjisvj ---
**Author:** ![Gravewalker](https://yyz2.discourse-cdn.com/free1/user_avatar/sef.discourse.group/gravewalker/32/12_2.png) [@Gravewalker](https://sef.discourse.group/u/Gravewalker)\ **Post date:** [July 10, 2021, 2:04am UTC](https://sef.discourse.group/t/using-apache-jmeter-to-test-the-scholarx-backend/308/11 "2021-07-10T02:04:09Z")
@piumal1999 nice work. Yup, let’s migrate the database to a PostgreSQL database. ---
**Author:** ![anjisvj](https://yyz2.discourse-cdn.com/free1/user_avatar/sef.discourse.group/anjisvj/32/9_2.png) [@anjisvj](https://sef.discourse.group/u/anjisvj)\ **Post date:** [July 12, 2021, 10:39am UTC](https://sef.discourse.group/t/using-apache-jmeter-to-test-the-scholarx-backend/308/12 "2021-07-12T10:39:15Z")
Let’s try Postgres then. ClearDB MySQL version is also old. Heroku gives you a Postgres DB hosted on an EC2 instance ---
**Author:** ![piumal1999](https://yyz2.discourse-cdn.com/free1/user_avatar/sef.discourse.group/piumal1999/32/274_2.png) [@piumal1999](https://sef.discourse.group/u/piumal1999)\ **Post date:** [July 14, 2021, 2:42pm UTC](https://sef.discourse.group/t/using-apache-jmeter-to-test-the-scholarx-backend/308/13 "2021-07-14T14:42:32Z")
Now the mvp is connected to a new postgresql database. [https://sef-scholarx.herokuapp.com/](https://sef-scholarx.herokuapp.com/) Now the query limit problem is fixed. ![image](https://global.discourse-cdn.com/free1/uploads/sef/original/1X/91bede9147a0b0ae8f218c0e83cb454d0d219833.png) ---
**Author:** ![anjisvj](https://yyz2.discourse-cdn.com/free1/user_avatar/sef.discourse.group/anjisvj/32/9_2.png) [@anjisvj](https://sef.discourse.group/u/anjisvj)\ **Post date:** [July 14, 2021, 3:48pm UTC](https://sef.discourse.group/t/using-apache-jmeter-to-test-the-scholarx-backend/308/14 "2021-07-14T15:48:55Z")
yay! Good job! @piumal1999 WIth that we completed the backend for the pilot right? ---
**Author:** ![piumal1999](https://yyz2.discourse-cdn.com/free1/user_avatar/sef.discourse.group/piumal1999/32/274_2.png) [@piumal1999](https://sef.discourse.group/u/piumal1999)\ **Post date:** [July 14, 2021, 4:22pm UTC](https://sef.discourse.group/t/using-apache-jmeter-to-test-the-scholarx-backend/308/15 "2021-07-14T16:22:44Z")
Yeah I think so. Let’s have a final QA as you said in [Final QA before launching ScholarX for the pilot - #3 by anjisvj](https://sef.discourse.group/t/final-qa-before-launching-scholarx-for-the-pilot/328/4) ---
**Author:** ![Gravewalker](https://yyz2.discourse-cdn.com/free1/user_avatar/sef.discourse.group/gravewalker/32/12_2.png) [@Gravewalker](https://sef.discourse.group/u/Gravewalker)\ **Post date:** [July 14, 2021, 5:09pm UTC](https://sef.discourse.group/t/using-apache-jmeter-to-test-the-scholarx-backend/308/16 "2021-07-14T17:09:58Z")
> [@piumal1999](#): > > Now the query limit problem is fixed. Nice, good job @piumal1999