This is my code: I am passing URL stored in a list and for each URL I parse it using 'html.parser'. I am looking for the term "livefyre"
for page in links:
req = requests.get(page, headers=hdr)
soup = BeautifulSoup(req.text, "html.parser")
for link in soup.find('div', attrs={"id" : "livefyre-comments"}):
print(len(link.get_text()))
This only outputs the div elements matching the specific "id" : "livefyre-comments". I want to search for all/any occurrences of "livefyre" anywhere on the HTML page. Please help.