Why Teams call analytics are critical to your entire business
Web hunting
1. Design of a Simple Intelligent Web Search
Agent
Amit Singh Dahal
G5638545 ITCS/M
2. semantic gap between the user‟s perception
of the search domain and the results
provided by search engines
most current Internet search engines suffer
from Recall and Precision problems
every search engine has an intelligent agent
elements of intelligent agent, some
algorithms required to develop intelligent
Web Search Agent
1
3. anything that gathers information or
performs some other service without your
immediate presence and on some regular
schedule
perceiving its environment through sensors
and acting upon that environment with
effectors/actuators
2
5. agent can generate all possible outcomes but
needs to filter from initial to current and
current to desired goal state
used to find the desired goals by the user
efficiently and effectively
two types of Algorithms
- Uninformed(blind search)
- Informed(heuristic search)
4
6. No information of number of steps or the
path cost from the current state to goal state
Searches include:
-depth first
-breadth first
-depth limiting
-iterative deepening
5
7. have the information about the goal
Information can be either estimated path cost
to the goal or estimated path cost away from
the goal
Information known as “heuristic”
Searches include:
-best first
-hill climbing
-A*(star)
6
8. Web hunter should know:
-the target
-initial state/point
-number of iterations of target to find
-time frame(constraint)
-search methods
For implementation:
-Socket programming
-HTTP/HTML
-Programming language
7
9. Initialization:
-set up all variables, structures and arrays
-base information about „hunt‟
-the target, initial state
-methods of searching
Perception:
-using knowledge to contact and retrieve
the information
-identify if target is present and path to
other URL locations
8
10. Action:
-takes all the information the system
knows and determines whether the goal
is met
-if met, the hunt is over else make
decision on where to go next
Effect:
-list the location of target and give
feedback to users
9
12. Testing the speed of searches and sorting
-time taken to search the target
-sorting the most common keywords first
Using loops, algorithms and timing functions
Searching the websites that has the maximum
number of hits by the user
11
13. Searching with high speeds produce burden
to the servers
Should be controlled using some timed waits
if Web Hunter make repeated queries to the
same servers
During timed waits, the hunter can visit
different servers and speed can be
maintained
12
14. Can integrate with Natural Language
Processing(NLP)
Machine learning implementation to
understand users behavior
In this way, improve the intelligence of Web
Based Agent
Can be modified to Document Hunter or
Record Hunter
13
15. Learned about:
-Web Hunting
-Role of Intelligent Web Search Agent
-Elements that need to be considered
when developing agent
with the help of some information and tools,
Web Hunting can help the users to achieve
the goals in quick succession
Keep in mind, “hunt safely”
14
