Instagram with my peninsula New and I have a little bit excited because with the talking about some great stuff including why should use automation what's a linear is

00:19

we'll actually do something so automation right here and talk about a page of design patterns and after that we'll quickly take a look at my tool against the and loss of of these we will go into why all open source is important and how you can level up your own open-source broke a portfolio and projects alike have

00:40

exact I'm and I'm currently is you in Germany and the creative and supply and if you have any questions feel free to contact me in any of the social media all directly we add the male of having checked and if you have any questions later today or tomorrow and then remember my phase like this white dot on the back of my head and it's really kind of to remember me also make sure to falling into the because I'll be posting slight cilia but now with the reasoning for the time let's get right into it so why should

01:16

you do automation I'm pretty sure every single 1 this room has at least 1 task you could connect the should alternate but they don't do they instead just do it by hand and an inverse Lee got some e-mails through in supply from social marketing companies that told me that asked me to automate their Instagram growth process because they have life uh 1 or 2 people sitting there and learning to clients accounts like pictures committing stuff and following people and they do it all by hand 8 hours a day and I think this is really ridiculous especially since ultimately stop it wasn't time 1 time effort we don't have to do it every time you might have to the uh made some slight adjustments but the big party stunned and it will save you time the more with the more often you have to do it especially since a lot of tasks are easier lot also made tuple then you actually think they are I just recently had to send e-mail to every single our contact in a CSV file and I could have come in there and do this by hand but instead I just wrote a script to go through sees violent send out an e-mail to everything so had time for more important stuff and speaking of Instagram or Twitter Facebook you could actually do something like this but I

02:44

as things get more complex this will

02:46

really exploding complexity and has

02:50

some disadvantages that's why we use a software only solution and that we will

02:55

use so when you and I don't know how many of you

02:58

have your cell your I couldn't make can OK that's quite a lot for those who don't know so there is actually a testing tool for front ends especially a web front ends but you can also be with front ends and by it's actually for testing you where that's for example you could test the flask the way that but it's also the very good for automating boring task which of course like stuff out there on Instagram is if you want to do it efficiently who has the already you've of reactive you gesso angular probably everyone and you can't uh crawl websites which evolve dynamically rendered like Facebook Twitter Instagram with with a simple request that's why Slim's also very interesting and since we're using Python you're

03:58

installing it is as easy as picking stole selenium and bring already to do some real Instagram automation so let's just take a look at what we have to do we have to start a new browser we have to navigate Instagram once we're on Instagram weekend bloggers in and went a lot and we can then start to click this little hockey items to actually like pictures then thanks using filling in here this is really

04:25

easy and we have to simply import direct fiber elements from so that from selenium and once we found that we can instantiate on you open your browser and in our case will just use the Chrome browser here but you can use Firefox over to counter documentation which browsers also supported and in the praises the can past the popular from driver this is not necessary but it gives you some Nice some security on different platforms and once we got uproars object ready we can then use the get method to navigate to any page in our case of course this is Instagram and once you're on Instagram we can then but character tonic 7 and well this leaves just here to make sure the browser doesn't flows instantly and we can see something and the prosody closes here for work at the next which is recommended but of new let's take a look at what happens

05:28

it so if we entered what if we stop the scriptural see that uh it opens a new

05:34

browser and many in so from management now from we can we see that there is but there is

05:45

no love in a form it's simply the sign of form and we have to make sure we get to on form it year

05:54

um and in this case there is this little log in link on the bottom of the page with which we can switch from the sign-up form to the login form and this is a really good

06:05

point you introduce a new concept here uh from selenium and is is finding and clicking elements so so we in order to find an

06:13

element in a page we can take a look at a page source and we have to find my signature of element in its case it's just and a tagged with the class of underscore and so on and the text of so we use that knowledge truly get the element we our browser element and for chromatin simply use the tools as well as so far in the wall have very nice that tools where you can search lens pretty efficiently if we translate this into could now we can use or

06:50

browser object to find element by xk path and for those who don't know exp off just a crude language for XML documents as additional in this example and takes part here is just like this that we will use the uh a link as an anchor and it is the so trailing which hasn't had a name of lobbying and text of organ industries and since we not have to modern element we can simply to trigger click on it which will actually

07:23

trigger click on the long and elements here and you can see this little flickering and this is just some unknown clicking on the link link to get us to the login form here yeah perfect so we now lot form and can lot of things we now have only have to send our username and password to the blog inform and once we've done that we are a lot in to insert them so let's check it out we 1st have to

07:51

again jective page source for our elements and will also introduce a new concept here which is called action changed and it does what it is called offer it simply change actions together and in this case we will just look for deform element and to log in button and if free yet and translated into protein and

08:13

we will get something like this and again we will use the browser element to find stuff I x path and for inputs it's just for platform flesh this let input which is of course a half for our input form and then we can instantiate new action chain element which will pass the browser and we don't have in access to some methods here which are moved to element clique keys and so perform which will trigger the whole action chain to execute and then we will take a look at the code from the bottom to from top bottom we 1st moved to the 1st input element which is the username within collector input element and this will really haven't uh more nature in natural feeling in a more human feeling than just trading links by Jasper and often we clicked into disuse elements we can then send all username to it in this case it's contacting jumbo and 1 system that we can move to the 2nd form element of course and send our password the after we entered are possible in username we can then simply creating new action chain and how of course with the element and I just added the 2nd action chain here because you have to see that you can also use it for 1 element only deepened have like 10 elements of 2 or 3 so again let's take a look at what this precipitous In this case it will just go

09:54

in and and use an impostor and click the voting element here and of course this

10:00

will trigger or it's to Lotus

10:03

he's the page and we're actually now looking successfully this is where it gets interesting because once looking and we

10:12

can instantly start clicking on the spot

10:15

the icons because we when we take a closer look at this from intradural course and phase we all have this feature here is this

10:24

little circular loading but uh loading social on the bottom of the page which will only get you content once we reach the bottom of the page so there's no and to begotten overhead here and this will just dynamically load new content and for a page and depending on how many images from stuff once we have to 1st get the total of that and make sure that we have enough data elements in the Pentagon wanted to 20 it's not a problem but if we want to 100 or 200 elements and our from page we will have to tackle this and again we will introduce new concept which are special keys and with the normal things along those and keys method we can simply send text like your username and password what we have to use special for stuff like delete benches space and In our case their home and the home and the entity which will trigger the page to go to the top of the page and then people triggered to be you got to the bottom and lets translate this into Korean this

11:31

will lead to something like this we have 2 important key to the special keys from solatium again and from then on we can simply use the keys and uses special special elements of which we have to define before it body element because we need an anchor to send our commands to we need an anchor to centers keys to and we can just use maybe things come over or something like that but we have to have a really solid solid element which will stay the same in this case it's a 40 element because it's of course the element you click on and you operate when you using the prosody of cells and then we can simply do a falling In this case we lose 3 times we will go to the bottom bottom of the page to the topic on the bottom of the page and will sleep 2 seconds in between each to make sure that the loading actually succeeds then we will get new content so we take a look at how this code gets

12:29

executed you will see that it moves to the bottom of the page and you move up again and

12:35

it's it's all about it just executes this 3

12:37

times in a row to make sure we got like

12:40

50 elements here to later like

12:45

so there is only 1 step left here which

12:47

is finding all the hot elements including them and you should not be able to do all by yourself because we introduce all the concepts here which are finding elements and cleaning elements course then if we take another look at the page source

13:04

here we just have to find hard items and find another signature you make sure that the signature is you need to make sure that we don't get another element on the page but exactly the 1 we need we want to and in this case well

13:24

use the class we use several classes to make really make sure that we will get all of these elements and all just move over here and we will do some printing to make sure that that we keep track of how many elements we always liked and if we enumerate grid we all of course and not to make sure we don't understand here we will get

13:51

something like this where we start our script from the start and we will see

13:56

everything timeline of so it starts a new

13:59

browser in any case Instagram once the Instagram world load in anyone's reluctant will then then

14:08

as we saw before start to move to the bottom of the page to the top again and once it's loaded

14:14

enough images it can finally start to like some pictures we'll

14:22

just wait few seconds you until it's done and

14:27

again you will then see the pages it's

14:30

really like something a little bit there you and you see that part items to that and is really just a very simple yeah automation of laughter liking for universities

14:43

and you can enter you can add your own

14:46

automation right now you can do filtering for friends and for example nature you don't like your own pictures you can add commenting

14:54

and you can add following 40 people

14:56

you actually see on the front page

14:58

or maybe even copied it

15:02

features and posting yourself or something like that

15:05

depends on what you read your depends on what you want to do with it but as his successors in

15:12

right now we heavily rely on the

15:15

page structure and this leads to some problems here because In succumbed Facebook and all these things pages where it would be interesting to do some automation on they change the page layout pretty recently and pretty often and this release 2 breaks in the codon and make sure that the code is not working very long and this is really annoying to fix but it's pretty quick actually and will look into some ideas on how to make this more maintainable and should make sure that our corpus so little morsel that the 2nd thing is different languages we saw that we rely on those modern text for searching for elements here and this is really bad for different countries where we have for example been Italy of course with the Italian ones in front of friends and so on so we can't we shouldn't rely on page and the page the language but selenium offers a really nice feature here which we with with which we can simply um force the prosody specific language and we can just enforce to use in which in this case and the most annoying problem is baby testing and all of these big companies of course to A B testing for those who don't know a b testing is just um rolling out features to a small group in a specific area like for example right we have a new feature we will tested in the on in Rimini and if the people in Rimini think this features wouldn't like this feature and there are no problems with this feature they will grow without incrementally to more and more people to make sure that every person on the planet if these big companies that gets a new feature and testing this is really annoying because we have to make sure you get it all information from the users or right there where are they located what's the what's the language of the system what system order using and all of this small in details which will or which can lead to a failed that is really you need is for that person n but selenium there Le règlement so pay adding and object design pattern for this page it's called page object design patterns and it of course is what it tells us it's an object for every page so we would have an object for of log page would have an object for our news that page and so on and it's really gets us more maintainable code because we only have 1 place to make our changes around the year I so in order to make changes right now are we would have to go through all all the whole car wouldn't make all the changes to the past and this is really really annoying and we don't want to do that it also reduces the complicated code because it has by nice layer of abstraction here which will move all the code to 1 or 1 big ball and in this cold and in this place there are will beyond logic it the thing is it adds to this effect of speculated as it's nice API layer which we can then use in our actual code to kind get some readable code and and if you add some comments and yeah um little hints were what we did here we can really get mice contributors and make sure that other contributors understand our code and can it contribute as well and since we're all real programmers you we will look at and you mail diagram of this page up to this but the same pattern you and it looks something like this well you know they're just

19:06

open it looks more like this and as you can see

19:10

here this is an example of the Google search page uh where every actually is an element in this case for example you i l is a tribute to the search box is a tribute and so on and every method reps selection for example navigation searching and today if you like about for example and of course it is actually a test pattern so we have an actual test last year for each page object in the test quoting here and but all of the matter if you got this right now we will look at an example with and so from here and we'll use that in from loading form for

19:53

this we all we want focus on on the extra patterns over the log in but the sign up link you will just focus on the username import of possible in for the input and along and got here and if again translate this into code we

20:11

get something like this where we have a class instilled in page and we in within the constructor constructor we will define every year educated every element has a set of recently for in this case it would be to use a box the possible to the log-in button and once we found all these elements we can then use it in the complete in the full class 1 so in our case we will implement a login method and it will get past the username and password and here in in the London method we can then instantiate a new action saying and schools through all the methods here and you want to element like we did before and 1 interesting thing about page job to design pattern is that it returns a new page object for a next page will move to so in our case as you saw when you log in we roll moved to the ones that in the the the page where our news is located so we will return a new instance of the and stuff page object in our case so that's not right now will switch over to in supply because you don't want to implement this sort yourself and think about how you can implement this and break your head about what what changes in suprematist sense something like that and I really so maze really decided to talk about this here today because this is actually what led to this talk and no I'm really excited how big it's gotten to what the open source community made out of this and it is called in support of course

21:47

and right now we have around 2 K stars on the top of which is I think a pretty nice for something like this especially since there are a lot of services out there which that Tulving down by the use of itself that where in there are services out there and this is really probably the open-source everyone can contribute and look at it and that even copy it if they want to I have had 32 contiguous which all from which 4 or in the core development team right now and we got a 424 it's of this project I hope there is a lot more to come because I want to keep his life and make sure that people can actually use it and this many contributors as many interests of people leads some to some very interesting stuff like at an active community means there is called for everyone so if someone submits an issue there is some if if I can't helping with this is you know like a hundred 200 people and maybe have had the same problem you can also comment on this stuff and uh and help me you fix all the problems and make sure that people actually can use this and to give you kind of impression what and stuff I could do you use a little list of features of course it can do following liking unfolding commenting you can like white hair and you can find your user specifically by location for example if you have and it's or uh and kind of shelf you can make sure that only people in your location will be modified but of course it works on the service can use digital ocean or any kind of ritual server to run and it's has offered support you can write it you can pose emoji used yeah and 1 interesting thing is clarified I don't know how many of you know clarify but it's a in which the smart Image Recognition API and with it you can make sure that they don't like any not safe work stuff or inappropriate stuff or maybe you're here from some country and you don't want to like Newt stuff for it depends on your preferences and you can really make sure that the common pictures you comment on you people you follow or not some kind of here and I think the API is pretty interesting

24:08

to you so this is what it looks like you just have to importance the fight from means the phi and in supply constructed you can then she just pass your username and password and after the look in you can do some settings in this case will just make sure that we don't injected any account has more than 2 and a half thousand forests and we want to do commenting on 10 per cent as well as following on 10 per cent we want to use a common school also nice um and we don't want to make sure that this this portal indexes our real friends so we can make sure that our friends or not included here and of course we can do some text search where a lot safer working naked or excluded and what is some of these bombing huge words are included in the lexical skip the paid the not like that Americans finally do some liking by tags for example in this case will just like 100 pictures for nature and head let's talk about it and so as a general and why it's important and I'm

25:13

pretty sure a lot of people in here already do all source and this is also but those who don't know but those who don't do it you really have to think it think about it if you have any project on your on your

25:25

hard drive that could be open source mind afterward I mean all in here already use open source for their projects but why not give something back to the community to get something laying around it doesn't

25:37

matter if you contributed to the topic at 11 make it open source for people to use it and I think this is really the most important thing to think about yeah it's also good for the developers to build a portfolio because people will find you on internet and if you've got free stuff to my contact you might want to work with you and even companies that are interested in you and this is where you get opportunities from and if you if you really look interesting they'll contact you so you don't have to send out any it any papers or something like that to match the Tripoli application of the company's to make it nice for you and if you're a little bit of motivation here is what open sourced from him by now I got 12 choristers and 1904 opera intention of from around the world as well as automata new context that contacted me through in the fight and I think this is these for itself if you got something that's that's interesting and maybe a little bit conf Russell it's really but just a matter of time until until people think about events and talk about it and really give you feedback on it and want to help out and tease job offers a context our from around the world including Sweden Sydney Argentinian the UK USA even from ir and Singapore also high and I think really speaks for itself and of course I got this talk from it which is that nice for me as well it to closes off here are some tips that I think are really important for you open source properly if you're interested interested in doing some open source staff make sure to include or to keep this information to make sure you're protestants here first one is x documentation I've seen so many projects out there who has not have have no real documentation and this is this really crucial I don't know it's annoying for people to write documentation but if people don't understand the project of don't even know what the features of your project . 2 this is just insane and how should they used this ability to to look into the pit code and I don't think so but they're really a few people actually do that the 2nd it's spread word you can use channels like I can use rated or death post something like this to make sure that people actually see Europe prohibition and you portfolio and your projects to make sure that you get feedback and and people to help you get people to talk about true and if nobody since nobody nobody cares about your project of course nobody will use it in the last 1 is supporting users and this is really time-consuming but it's really worth it it's a lot of work but it's worth it and if somebody uses ritual and if he stops using the true because they couldn't get it to work they will never come back and you know you lost a chance to get a user so what what's

29:02

next for 4 is applied in this project we want you to take a look at that maybe contributed users give us the fact and maybe add some issues of like that we will we want to and the just like we smarter following and that more human and automation and that like I

29:22

said if you have any questions or recommendations offers anything you just make sure to contact me in any of the social media or Gmail and I'll be happy to chat later or through 1 of these channels thank you for your attention and

Inhaltliche Metadaten

Automating Instagram with Python and Selenium [EuroPython 2017 - Talk - 2017-07-12 - Anfiteatro 1] [Rimini, Italy] In this talk about using Selenium and Python for social media automation you will get some insights into growing your own open source projects into tools that really get used and maintained by a community. You will get an idea of what pitfalls you have to face when working with something as fast changing as Instagram and how to best tackle this. You'll learn what the ""Page Object"" design pattern is, what it is useful for and why we use it. Of course we will also talk about OpenSource and why it's important. A lot of the talk will be based on my article on InstaPy published at the freeCodeCamp publication on Medium: https://medium.freecodecamp.com/my-open-source-instagram-bot-got-me-2-500-real-followers-for-5-in-server-costs-e40491358340 So, who is this talk suited for? If you're just starting out with python, are interested in automation or simply like to see a fun and interesting little open source project, I'd love to see you at my talk and hear your ideas and opinions about it