browser_automation package¶

browser_automation.browser_manipulation module¶

Module: browser_manipulation This module contains functions and classes related to manipulation of browser. such as get html source code, fill_input_on_webpage, write_in_input_box, click_on_button. To do: change the exceptions to selenium exceptions by importing the selenium exceptions from selenium.common.exceptions

class browser_automation.browser_manipulation.Interactions(custom_browser_options_json_path=None)[source]¶

Bases: browser_automation.browser_setup.Setup

This is class for doing manipulations with web elements using selenium drivers.

articles_gathering_loop(articles_name_list: list, initial_search_link: str, pdf_button_locator_element_xpath: str, destination_folder_path: str, scroll: bool = False, open_button: bool = False)[source]¶

This is a sample loop for gathering the data(research articles).

Parameters

articles_name_list (list) – This list contains the name of search argument in website search box.
initial_search_link (str) – This is the link where we have to get data.
pdf_button_locator_element_xpath (str) – This is the xpath address of the element of data download button.
destination_folder_path (str) – This is the path of the folder where we want to move our downloaded data.
scroll (bool) – This is the option to switch on scroll on the page.
open_button (bool) – This is the additional dialogue box after clicking on download button.

Returns

Return type

None

button_click_by_xpath(button_element_xpath: str)[source]¶

ButtonElementXpath used to find and click on the button

Parameters: button_element_xpath (str) – This is the address of locator type ‘XPATH’ for button element.
Returns
Return type: None

change_window()[source]¶

It changes the current window to last window in browser

Returns
Return type: None

click_on_button(element_locator_type: str, element_locator_path: str)[source]¶

click on button on web by searching element_locator_path. exceptions silenced

Parameters

element_locator_type (str) – these by selenium methods to locate web element. This function contains CLASS_NAME, CSS_SELECTOR, ID, LINK_TEXT, NAME, PARTIAL_LINK_TEXT, TAG_NAME, XPATH.
element_locator_path (str) – path of above given element_locator_type

Returns

Return type

None

create_article_weblink_and_browse(initial_search_link: str, search_argument: str)[source]¶

Some website can directly take search argument after their link. so this function put the search argument after initial_search_link.

Parameters

initial_search_link (str) – This is the web link that can be used to append search argument and return required data page.
search_argument (str) – This can be any search string such as article name in research database.

Returns

Return type

None

driver_wait(time_duration: float)[source]¶

This makes the driver idle for time_duration.

Parameters: time_duration (float) – This is the time for which we want out driver to stay idle to minimize load on web server.
Returns: Returns wait time of time_duration
Return type: function

fill_input_on_webpage(word_seen_in_website: str, element_name: str, input_answer: str)[source]¶

this function fill the input box on webpage with the help of wordSeenInWebsite, with the inputAnswer.

Parameters

word_seen_in_website (str) – word that you see in input box on website
element_name (str) – ID, Name, Class etc. of html element
input_answer (str) – answer that you want to put in the input box of webpage

Returns

Return type

None

find_web_element(element_locator_type: str, element_locator_path: str)[source]¶

click on button on web by searching element_locator_path

Parameters

element_locator_type (str) –

these by selenium methods to locate web element. This function contains
CLASS_NAME, CSS_SELECTOR, ID, LINK_TEXT, NAME, PARTIAL_LINK_TEXT, TAG_NAME, XPATH.
element_locator_path (str) – This is the address of given element_locator_type.

Returns

selenium.webdriver.remote.webelement.WebElement, It could be input_box or button etc.

Return type

object

get_current_webpage()[source]¶

html_source_code: complete html of page

Returns: It is html source code of current webpage.
Return type: str

input_box_by_name(name: str, answer: str)[source]¶

Input answer in input box by searching element Name

Parameters

name (str) – This is element locator Name, visit - https://selenium-python.readthedocs.io/
answer (str) – This is the input string you want to put in input box on webpage.

Returns

Return type

None

input_box_by_xpath(xpath_element: str, answer: str)[source]¶

Input answer in input box by searching element Name

Parameters

xpath_element (str) – This is the address in xpath format, visit - https://selenium-python.readthedocs.io/
answer (str) – This is the input string you want to put in input box on webpage.

Returns

Return type

None

multiple_try_to_find_web_element(list_of_element_locator_types_with_addresses: list) → object[source]¶

This functions uses multiple try statements to find_web_element

Parameters: list_of_element_locator_types_with_addresses (str) – This is the list with element_locator_types such as CLASS_NAME,CSS_SELECTOR, ID, LINK_TEXT, NAME, PARTIAL_LINK_TEXT, TAG_NAME, XPATH with respective addresses. example- [“id”, “result-header-1”, “xpath”, “//h3[@id=’result-header-1’]”]
Returns: The web_element object from selenium driver. selenium.webdriver.remote.webelement.WebElement
Return type: object

scroll_page(scroll_until: int = 4000, scroll_height: int = 200)[source]¶

this scroll the page to load all of details

Parameters

scroll_until (int) – This is the total amount of scrolling.
scroll_height (int) – This is how much scroll is happen in one go.

Returns

Return type

None

try_element_address(element_locator_type: str, element_locator_path: str)[source]¶

This is try except block for searching element via element_locator_type, element_locator_path

Parameters

element_locator_type (str) –

These by selenium methods to locate web element. This function contains
CLASS_NAME, CSS_SELECTOR, ID, LINK_TEXT, NAME, PARTIAL_LINK_TEXT, TAG_NAME, XPATH.
element_locator_path (str) – This is the address of given element_locator_type.

Returns

This is bool representing if web_element found or not. object - web element from selenium. selenium.webdriver.remote.webelement.WebElement

Return type

tuple

web_element_multiple_try_to_click_or_write(list_of_element_locator_types_with_addresses: list, write_something=None)[source]¶

This function takes list of element_locator types with addresses to click or write in input_box. Note - This is worst case of finding different addresses for same web element on website

Parameters

list_of_element_locator_types_with_addresses (str) – This is the list with element_locator_types such as CLASS_NAME,CSS_SELECTOR, ID, LINK_TEXT, NAME, PARTIAL_LINK_TEXT, TAG_NAME, XPATH with respective addresses. example- [“id”, “result-header-1”, “xpath”, “//h3[@id=’result-header-1’]”]
write_something (str) – default is None which signify that click() function will be executed. if you want to write something in found web element, please replace None to your input string.

Returns

Return type

None

website_login(url: str, username_element_locator_type: str, username_element_locator_path: str, username: str, password_element_locator_type: str, password_element_locator_path: str, password: str, login_button_element_locator_type: str, login_button_element_locator_path: str, login_option_element_locator_type=None, login_option_element_locator_path: Optional[str] = None)[source]¶

login into website using these arguments Args: driver: url: username_element_name: username: this is the username of person already sign up on website password: password for above username of account

Parameters

url (str) – This is link for website.
username_element_locator_type (str) – This is locator type for username_element for username.
username_element_locator_path (str) – This is the address of locator type.
username (str) – This is the input username for given website.
password_element_locator_type (str) – This is locator type for password_element.
password_element_locator_path (str) – This is address of locator type for password.
password (str) – This is the input password for given website.
login_button_element_locator_type (str) – This is locator type for login button_element.
login_button_element_locator_path (str) – This is the address of locator type for login_button.
login_option_element_locator_type (str) – This is locator type for login_option button_element.
login_option_element_locator_path (str) – This is the address of locator type for login_option.

Returns

Return type

None

write_in_input_box(element_locator_type: str, element_locator_path: str, input_answer: str)[source]¶

Input answer in input box by searching element_locator_path

Parameters

element_locator_type (str) – these by selenium methods to locate web element. This function contains CLASS_NAME, CSS_SELECTOR, ID, LINK_TEXT, NAME, PARTIAL_LINK_TEXT, TAG_NAME, XPATH.
element_locator_path (str) – path of above given element_locator_type
input_answer (str) – This is the input string you want to put in input box on webpage.

Returns

Return type

None

browser_automation.browser_setup module¶

Module: browser_setup This is the sole of the python package. It helps setup the required browser and with custom options.

class browser_automation.browser_setup.CustomOptions(custom_browser_options_json_path=None)[source]¶

Bases: object

get_browser_custom_options_dict()[source]¶

returns browser custom options dict.

Returns: This provide the list of current browser options.
Return type: dict

get_default_custom_options_json(output_file_path: str = 'custom_options.json')[source]¶

This function write a sample json file for the user to edit for custom options.

Parameters: output_file_path (str) – This is the path for writing the sample json file.
Returns
Return type: None

class browser_automation.browser_setup.Setup(custom_browser_options_json_path=None)[source]¶: Bases: object

browser_automation.locate_elements module¶

Module: locate_element This module help in getting the address for selenium.webdriver.remote.webelement.WebElement Note - Try using the extension for finding the web elements rather than this module. Use this as the last resort. Examples - My favorite https://selectorshub.com/ This website contains full documentation and videos. Just provide the extension filepath in browser setup options. Other example - https://www.ranorex.com/selocity/browser-extension/, ‘chropath’ https://chrome.google.com/webstore/detail/chropath/ljngjbnaijcbncmcnjfhigebomdlkcjo

class browser_automation.locate_elements.LocateElements(html_source_code)[source]¶

Bases: object

get_element_name_value_regex(element_name: str, word_seen_in_website: str)[source]¶

This get element locator type and its value or address.

Parameters

element_name (str) – ID, Name, Class etc. of html element
word_seen_in_website (str) – snippet of Html which has your element Name

Returns

This contains list of element type and address.

Return type

list

get_element_value_from_html(word_seen_in_website: str, element_name: str) → list[source]¶

This return only the address of web element of required element type.

Parameters

word_seen_in_website (str) – word that you see in input box on website
element_name (str) – ID, Name, Class etc. of html element

Returns

get element value ex: ember51

Return type

list

get_link_via_regex()[source]¶

Specific function for IEEE xplore pdf open button

Returns: link of some sort from website
Return type: str

snippet_html(word_seen_in_website: str) → str[source]¶

Get the important chunk of html source code

Parameters: word_seen_in_website (str) – word that you see in input box on website
Returns: snippet of Html which has your element Name
Return type: str

browser_automation.locate_elements.get_element_value(element_name_value: str) → list[source]¶

This function uses regex patter to find the element value from element name and value

Parameters: element_name_value (str) – element name and element value ex: id=”ember51”
Returns: get element value ex: ember51
Return type: list

browser_automation.os_json_utils module¶

Module: os_json_utils This module contains functions related to getting or writing files from os. This also include json read and write functions for use in browser options. This also contains sleeping and waiting for the file to be downloaded.

class browser_automation.os_json_utils.DownloadUtil(download_dir: str)[source]¶

Bases: object

download_completion_wait(timeout: int = 300, files_counts: Optional[int] = None)[source]¶

Wait for downloads to finish with a specified timeout. author: https://stackoverflow.com/questions/34338897/python-selenium-find-out-when-a-download-has-completed

Parameters

timeout (int) – How many seconds to wait until timing out.
files_counts (str) – If provided, also wait for the expected number of files.

Returns

Return type

None

file_rename_and_moving(destination_folder_path: str, new_name: Optional[str] = None)[source]¶

This function waits with timeout option until file is downloaded in origin folder. then rename and move it to destination folder.

Parameters

destination_folder_path (str) – This is the address of destination folder where we want to move files from download location.
new_name (str) – This is the new name of file we just downloaded and want to change previous name.

Returns

Return type

None

class browser_automation.os_json_utils.RandomSleep(lower_time_limit: float = 5, upper_time_limit: float = 15.3)[source]¶

Bases: object

random_time()[source]¶

this gives random_time between lower_time_limit and upper_time_limit.

Returns: This contains random value between lower and upper time limit.
Return type: float

sleep_after_n_monotonous_work(n: int, monotonous_work: int, random_sleep_multiple: float = 15)[source]¶

sleep every n applications

Parameters

n (int) – This signify number of loop or repetitive tries.
monotonous_work (int) – temporary variable to count the repetitive tries.
random_sleep_multiple (float) – This is the value by which you want to multiply the random time.

Returns

Return type

None

sleep_for_random_time()[source]¶

this sleep for some time

Returns
Return type: None

browser_automation.os_json_utils.avoid_lock()[source]¶

this uses mouse and keyboard to act human is using the computer

Returns
Return type: None

browser_automation.os_json_utils.config_file_path(config_name: str = 'config.json')[source]¶

This is useful if config file is in grandparent directory.

Parameters: config_name (str) – This is the name of json config file.
Returns: path of the config file at grandparent directory.
Return type: str

browser_automation.os_json_utils.finding_grandparent_dir_from_cwd()[source]¶

This provide the grand parent directory from current working directory.

Returns: grand parent directory name
Return type: str

browser_automation.os_json_utils.get_download_location_path() → str[source]¶

Get the default download location based on operating system. Written for chrome.

Returns: default download location of browser.
Return type: str

browser_automation.os_json_utils.list_to_text_file(filename: str, list_name: str, permission: str = 'w')[source]¶

This converts list to text file and put each element in new line.

Parameters

filename (str) – This is the name to be given for text file.
list_name (list) – This is the python data structure list which contains some data.
permission (str) – These are the os permissions given for the file. check more lemma_info on python library ‘os’.

Returns

Return type

None

browser_automation.os_json_utils.load_text_file_to_list(file_path: str, permission: str = 'r')[source]¶

This converts text file to list and put each line in list as single element. get first line of text file by list[0].

Parameters

file_path (str) – This is the name to be given for text file.
permission (str) – These are the os permissions given for the file. check more lemma_info on python library ‘os’.

Returns

This contains all lines loaded into list with one line per list element. [first line, second line,…. ]

Return type

list

browser_automation.os_json_utils.read_json_file_from_path(json_file_path: str) → dict[source]¶

Read the json file from the path given. Convert json file data to the python dictionary.

Parameters: json_file_path (str) – This is the json file path which is needed to be converted.
Returns: This is the data in dict format converted from json file.
Return type: dict

browser_automation.os_json_utils.replace_symbols_with_space(string: str) → str[source]¶

replace symbols in string with spaces. Example - ‘df%$df’ -> ‘df df’

Parameters: string (str) – This is input word string which contains unwanted symbols.
Returns: This is cleaned string from symbols and contains only alpha characters and all lowercase character string.
Return type: str

browser_automation.os_json_utils.sleep(time_duration: float)[source]¶

this function sleep for given Time

Parameters: time_duration (float) – This is the duration of time, you want script to sleep.
Returns
Return type: None

browser_automation.os_json_utils.write_json_file_with_dict(output_file_path: str, input_dict: dict)[source]¶

Write json file at output_file_path with the help of input dictionary.

Parameters

output_file_path (str) – This is the path of output file we want, if only name is provided then it will export json to the script path.
input_dict (dict) – This is the python dictionary which we want to be saved in json file format.

Returns

Function doesn’t return anything but write a json file at output_file_path.

Return type

None