https://www.youtube.com/watch?v=dhxqn6RZkm0 ================================================================================ Use Request and BeautifulSoup for crawling JavaScript based web - See Network tab - See memCondListAjax.do - /home/young/Pictures/2019_05_21_08:49:20.png - See Hearders tab - /home/young/Pictures/2019_05_21_08:49:57.png - You can see POST method - When parameters is passed to the server, parameters are not attatched onto URL, but parameters are passed as object into the server - See Form data - /home/young/Pictures/2019_05_21_08:51:14.png - note that currentPage, rowPerPage (number of pictures per page) - Copy parameters and paste them into URL as query string - /home/young/Pictures/2019_05_21_08:53:08.png Rule: 1 page (1) has 6 pictures 2 page (7) has 6 pictures 3 page (13) has 6 pictures Trick: Display all pictures in 1 page Querystring: url?currentPage=1&rowPerPage=1000 ================================================================================ Let's crawl specific data (unique ID number) * Code