<p>I would like to implement a search engine which should crawl a set of web sites, extract specific information from the pages and create full-text index of that specific information.</p>

<p>It seems to me that Xapian could be a good choice for the search engine library.</p>

<p>What are the options for a crawler/parser to integrate with Xapian?</p>

<p>Would Solr be a better choice than Xapian to integrate with open source crawlers/parsers?</p>

First of all you should set authorization cookie:
```
```
FormsAuth.SetAuthCookie(UserName, RememberMe);
```
```
After this you should return RedirectToAction result:
```
```
return RedirectToAction(actionName, controllerName);
```
```
Or Redirect result:
```
```
return Redirect(url);
```
```
When you use "return Redirect(url);" it uses Response.Redirect(url, false) method for redirecting so it must work. May be you not returning this action result, please check this.
"FormsAuthentication.RedirectFromLoginPage()" will redirect to login page when you didn't set authorization cookie or when your login page is a home (default) page of the application -
The RedirectFromLoginPage method redirects to the URL specified in the query string using the ReturnURL variable name. For example, in the URL
http://www.contoso.com/login.aspx?ReturnUrl=caller.aspx
, the RedirectFromLoginPage method redirects tothe return URL caller.aspx. If the ReturnURL variable does not exist, the RedirectFromLoginPage method redirects to the URL in the DefaultUrl property.
Via MSDN