A is incorrect
A is incorrect because artificial intelligence may be involved in other processes of search engines, but not specifically for discovering and indexing web pages. Hint: The term used describes an automated process that follows links.
B is incorrect
B is incorrect because there is no mention of "hyperlink aggregators" in the course content. Hint: The correct term is mentioned directly in the context of how search engines work.
D is incorrect
D is incorrect because data mining algorithms might analyze data, but they are not used for the specific task of discovering and indexing web pages. Hint: The correct answer describes the agents that 'crawl' the web.