Price Tracker Application with Django | Track Discounts
In this tutorial we are going to build price tracker application which will notify us for discounts.
Creating complex projects is the key to learn fast! If you are following me for a while, you already know I like complex stuff, so we are going to use RabbitMQ + Celery + BeautifulSoup + Django for creating this app.
Alright! Let's Start!
Celery is the best choice for doing background task processing in the Python/Django ecosystem. It has a simple and clear API, and it integrates beautifully with Django. So, we are using Celery to handle the time-consuming tasks by passing them to queue to be executed in the background and always keep the server ready to respond to new requests.
Celery requires a solution to send and receive messages; usually this comes in the form of a separate service called a message broker. We will be configuring celery to use the RabbitMQ messaging system, as it provides robust, stable performance and interacts well with celery.
We can install RabbitMQ through Ubuntu’s repositories by following command:
Then enable and start the RabbitMQ service:
Well, create a new Django project named pricetracker and app named tracker
Install following dependencies:
Once installation completed, add the CELERY_BROKER_URL configuration to the settings.py file:
Then, create celery.py inside your project.celery.py
We are setting the default Django settings module for the 'celery' program and loading task modules from all registered Django app configs.
Now inside your __init__.py import the celery:
This will make sure our Celery app loaded every time Django starts.
To run the celery use following command at the end of this tutorial:
celery -A pricetracker worker -l info
Note that, you have to open another terminal and run celery there so you will have two terminals. First for Django server, second one is for celery
Now, Let's create our modelmodels.py
We are going to crawl eBay. User will enter URL of specific item and requested price.
So, let's create a form for that:forms.py
We will use beautifulsoup to crawl price and title of item in given URL. After data crawled we have to convert price to float and create new object in database.views.py
strip() removes spaces at the beginning and at the end of the string and replace() method replaces a specified phrase with another specified phrase. So, we used these methods to get clean price and title of item.
Once data crawled successfully, it is time to create new object in database. We will use this function in form submission to crawl title and price of new item.views.py
Great! Now, we need to crawl the data for all objects continuously to be aware of discounts. If we do this without celery the server connection will timeout which means that a server is taking too long to reply to a data request and our application will crash.
Create tasks.py in your app and let's handle it with celery tasks.tasks.py
@shared_task will create the independent instance of the task for each app, making task reusable. This makes the @shared_task decorator useful for libraries and reusable apps, since they will not have access to the app of the user.
We are simply crawling data every 15 seconds and comparing last price with requested price. If last price is smaller than requested price then we are updating the discount price field.
What if price will increase again?
Great! Now, it will possible to track discounts properly. You can add one more function which will detect closer prices and notify user about it. For instance, if item price is 100$ and requested price is 97$. But let's keep it simple for now.
Finally, we can create our template.tracker.html
Well, you can improve the project by adding email functionality so Django will send email about discounts. Take a look How to Send Email in a Django App
Feel free to contribute the project :)
You can clone this project from my GitHub repository below
That's all for now! Make sure you are following me on social media and if you find it useful please share Reverse Python with your friends. See you in next post DEVs!