AWS Webcast - Location Based Search

Building Location-Based Search
with

Amazon CloudSearch

Tom Hill
April 3, 2013

© 2013 Amazon.com, Inc. and its affiliates. All rights reserved. May not be copied, modified or distributed in whole or in part without the express consent of Amazon.com, Inc.

Agenda
!   What is CloudSearch
!   What is Location-Based search?
!   Computing distance
!   Location-Based Search in CloudSearch
!   Sample App


Housekeeping
!   Type questions into GoToMeeting window
•  We'll get to them at the end.
!   Recording will be on the AWS YouTube channel
!   Slides will be on Slideshare.net
•  A link to slides will be mailed to all participants
!   You can maximize the shared screen's window


What is CloudSearch?

What is CloudSearch
!   Amazon CloudSearch is a fully-managed search service
!   Easy to add search functionality to your application
!   Fast and highly scalable
!   Supports search features like
•  Faceting
•  Synonyms, stopwords
•  Ranking


How is CloudSearch Used
!   Create/Configure a "search domain"
!   Post documents via HTTP
!   Search via HTTP
•  results as XML or JSON
!   Scales automatically

http://search-‐DOMAIN-‐XYZ.REGION.cloudsearch.amazonaws.com/2011-‐02-‐01/search?q=cat


More Information
!   Webinar
•  'Getting Started With Amazon CloudSearch"
!   April 17, 9:00am PT / 12:00pm ET
!   Register at http://bit.ly/11WZRAZ


What is Location Based Search?

What is Location-Based Search?
!   Using location & distance as factors in search


Why Do You Care?
!   People care about things near them.
•  Pizza, Classified Ads, etc.
•  Find a Doctor, Lawyer,…
!   Mobile is a key driver


How do we use distance?
!   Limit to an Area
•  Box
•  Circle

!   Sort by Distance

!   Include distance in ranking
•  combine text relevance and distance


Computing Distance

Computing Distance
!   Many Formulas
•  Rectangular distance
•  Equi-rectangular projection
•  Spherical Law of Cosines
•  Haversine Formula
!   Speed Vs. Accuracy
•  Speed: Rectangular distance
•  Accuracy: Haversine Formula


Why So Many Ways to Compute Distance?

The earth isn't flat! It isn't a sphere either.


Is the Earth Flat?
!   If it's flat
(x − x2)2 + (y − y2)2
!   If it's not flat
# &
2 # φ 2 − φ1 & 2 # λ2 − λ1 &
2r × arcsin % sin %
% ( + cos(φ1 )cos(φ2 )sin % ((
$ $ 2 ' $ 2 '( '


How Much Difference?
!   Distances between some cities
•  Fort Lauderdale, FL to New Haven, CT
•  Haversine: 1813

•  Rectangular: 1872

•  Difference:

3%

•  Bangor,
ME
to
Adak,
AK

•  Haversine: 7244

•  Rectangular: 12016

•  Difference:

66%


Difference in Detail
!   Four computations
•  Rectangular Distance
•  EquiRectangular Projection
•  Spherical Law of Cosines
•  Haversine

Haversine Cosines EuqiRect Rect - CosErr EquErr RecErr
1812.54997 1812.54997 1814.38516 1871.77660 - 0.000 0.001 0.033 Fort Lauderdale, FL to New Haven, CT
7244.45661 7244.45661 8008.74698 12015.96646 - 0.000 0.106 0.659 Bangor, ME to Adak,AK

Same Close Not so close


How Much Accuracy?

!   "Pizza, 1 Mile"
! Haversine is more accurate
•  If you are a bird
!   Any distance is approximate


Which one is the Right Choice?

!   As usual: "It depends"
!   Factors
•  Desired query speed
•  Index size
•  Accuracy needed
!   Start with Equirectangular Projection
•  Test


Location-Based Search in CloudSearch

How to Compute Distance in CloudSearch?

!   Rank Expressions
•  Computations run for each matching document
!   Can be used for
•  Sorting
•  Influencing Scoring


Two Types of Rank Expressions
!   Static Rank Expressions
•  Computation based on values in index
!   Query Time Rank Expressions
•  Allow including parameters at run time.
•  e.g. latitude, longitude.


Converting to Rank Expression
!   Rank expressions have JavaScript-like syntax
!   Most math functions
•  log2, log10,sin, cos, atan, min, max, etc.
!   So this distance formula:
(x − x2)2 + (y − y2)2
!   Becomes
•  sqrt(pow(lat-userlat),2)+pow(lon-userlon),2))


Query Time Rank Expressions
!   Define Rank Expression
•  &rank-NAME=EXPRESSION
•  &rank-geo=sqrt(pow(lat-userlat),2)+pow(lon-userlon),2))
!   Select Rank Expression
•  &rank=NAME
•  &rank=geo

http://searchendpoint?q=creek&rank=geo&rank-geo=sqrt(pow(la1-123),2)+pow(lo1-456),2))


Influencing Scoring
!   Include both text relevance and distance:

&rank-geo=(1000-text_relevance) + sqrt(pow(la1-ulat),2)+pow(lo1-
ulon),2))

!   Relevance: Higher is better
!   Distance: Lower is better


Combining Relevance and Distance
!   Which is more important distance or relevance?
•  By how much?
!   Relative Weight
•  N * text_relevance + M * distance
!   That's where the art comes in
•  Will vary by your application
•  Test & tune, and test again
•  Rank expression comparator


Compare Rank Expressions


Storing Data
!   CloudSearch supports unsigned integers
!   Have to convert latitude, longitude to positive ranges
•  latitude + 90
•  longitude + 180
!   Have to store as integers; need to scale
•  latitude = (latitude + 90) * 1000
•  longitude= (longitude+ 180) * 1000


Performance
!   Don't query the whole world
•  Can limit by literals or numeric fields.
•  Literals are more efficient for limits.
!   Limit Options
•  Literal
•  &bq=state:'CA'
•  &bq=zip:'94402'
•  Numeric
•  &bq=(and latitude:40..50 longitude:80..85)


Performance Measures
Why you don't want query the whole world!

GeoMethod

TextRel

Limits

Queries

Seconds

QTimeMS

Threads
CompletedQ

AveHits

NONE

false

10

6.2255

622

1

10

8345450.00

CARTESIAN

false

10

15.6064

1560

1

10

8345450.00

EQUI

false

10

19.7106

1971

1

10

8345450.00

COSINES

false

10

27.4968

2749

1

10

8345450.00

HAVERSINE

false

10

31.2595

3125

1

10

8345450.00

NONE

false

Numeric

10

9.1758

917

1

10

3807.00

CARTESIAN

false

Numeric

10

9.0255

902

1

10

3807.00

EQUI

false

Numeric

10

9.1158

911

1

10

3807.00

COSINES

false

Numeric

10

9.8321

983

1

10

3807.00

HAVERSINE

false

Numeric

10

9.1272

912

1

10

3807.00

NONE

false

literal

10

0.8254

82

1

10

3781.00

CARTESIAN

false

literal

10

0.5936

59

1

10

3781.00

EQUI

false

literal

10

0.6173

61

1

10

3781.00

COSINES

false

literal

10

0.5916

59

1

10

3781.00

HAVERSINE

false

literal

10

0.6289

62

1

10

3781.00


Geo-Spatial Demo Application

Demo Application Structure

HTML Page
Ajax Server CloudSearch
Javascript
(Tomcat)


Demo Implementation
!   JavaScript
•  Ajax
•  JQuery
•  Google Maps API
!   Tomcat Server
•  Java
•  Just for forwarding of requests
•  Because XSS, that's why.


Querying CloudSearch
$.ajax({

url
:
"searchx",

data
:
{

'q'
:
currentQuery,

'domain'
:
"geoname25",

'return-‐fields'
:
returnFields.join(),

"rank"
:
"geo",

"rank-‐geo"
:
"Math.sqrt(Math.pow(Math.abs(doc.latitude_90-‐
12539),2)
+

Math.pow(Math.abs(doc.longitude_180-‐
5784),2))"

},

dataType
:
"json",

success
:
function(data)
{

var
hits
=
data['hits'];

displaySearchResults(hits['hit'],
hits['found']);

populateMap(hits['hit']);

}

});

Getting Latitude & Longitude

Getting Latitude & Longitude
!   What if you don't know the latitude & longitude?
! GeoCoding Services
•  Many services
•  Some free tiers
•  May have restrictions
•  Google, Bing, Yahoo, MapQuest, ArcGis, …


Wrap Up

Recap
!   Local search is a component of many applications
!   CloudSearch supports local search
•  Using rank expressions
! GeoCoding services


Thanks for Coming!
!   Data
•  http://www.geonames.org/export/
!   Slides
•  Slideshare.net soon. We'll send you a link.
!   Sample Code
•  Talk to me. (tomhill@amazon.com)
!   Computations
•  http://www.movable-type.co.uk/scripts/latlong.html
•  wikipedia


Any Questions?


Thanks for Coming!


Use Zip Code, City, State as a proxy
!   Multiple ways to select, map, zip, current location


Radians vs Degrees
!   Should note somewhere that all but rectangular distance
require radians.


Querying Data
!   Java/Javascript
•  Fields are latitude_90, longitude_180
•  user location is userlat, userlon

!   simple distance
rank
=
"Math.sqrt(Math.pow(Math.abs(latitude_90-‐("
+
userlat
+
")),
2)+Math.pow(Math.abs(longitude_180-‐("
+
userlon
+
")),2))";

!   Spherical Law of Cosines
rank = "6371*Math.acos(Math.sin(" + userlat + ") * Math.sin(lat_rad/" + scale + ") +
Math.cos(" + userlat + ") * Math.cos(lat_rad/" + scale + ") * Math.cos((lon_rad/" + scale +
") - " + userlon + ") )";


What CloudSearch Doesn't Do


Issues with assuming
the earth is flat.


AWS Webcast - Location Based Search

Recommandé

Recommandé

Contenu connexe

Tendances

Tendances (20)

Similaire à AWS Webcast - Location Based Search

Similaire à AWS Webcast - Location Based Search (20)

Plus de Amazon Web Services

Plus de Amazon Web Services (20)

Dernier

Dernier (20)

AWS Webcast - Location Based Search