Category: RavenDB

RavenDB – Exporting and Importing using Smuggler

badge1Smuggler is a tool for RavenDB which is used for exporting and importing data – today at work I was shown a nice way to use it to take production data out and import it locally for just one collection of documents – something which might be useful and I know I will forget the syntax I the future for so blogging about it helps me know remember it and I can always read here again in the future if and more likely when I do forget it.

The syntax for exporting a collection called Categories would be as follows:-

smuggler

What are we doing here, well we are saying use Smuggler to export from the Catalogue Tenants database a dump file – we are also using metadata so that we are only exporting a collection of documents called Categories and then only the latest version so no revisions, and lastly only export the Documents so no Indexes or any other types of documents.

I wanted to then import this into my local instance of RavenDB so the following command does just that:-

smuggler2

This will import the Categories and override any existing Categories in my local RavenDB instance.

You can read more about smuggler here.

Hope someone finds that useful.




My DotNetCurry Magazine Articles

Over the last couple of months I have been privileged to write a couple of articles for the free .Net Developer Magazine brought to you by
DotNetCurry.

Its been my first attempt at writing for a magazine and has been a lot of fun, maybe in the future I can write more articles.

Please enjoy and give them a read – and sign up for the magazine – its FREE!

dnc-nov-thumb
Create a Snappy UI with KnockoutJS -> Issue 3
This article explores KnockoutJS and what KnockoutJS gives you as a developer.

dnc-jan-thumb
Web Essentials for Visual Studio 2012 -> Issue 4
This article explores the Visual Studio 2012 Web Essential add-on.

dnc-mar

What’s New in RavenDB 2.0 -> Issue 5
This article explores RavenDB 2.0

Take a look at these articles and hopefully you will learn something new – Gregor.




RavenDB – Safe By Default

One of RavenDB’s unique features is something called safe by default – this means that the database is configured to stop users querying for large amounts of data which is never a good idea, to have a query return thousands of records is inefficient and can also take a long time.

With RavenDB safe by default limits the number of records by default on the client side to 128 (by default means its configurable) if you don’t make use of Take(). Server side you can use .Take(1024) if you want to go up to the limit configured by the server, if you try to bring back more than 1024 then it will default to 1024 results returned if you need to return more data than that then you should be asking yourself why?, page your data if you need to return more than this number of records.

Safe by default is attempting to stop a developer from writing poor queries, there is nothing to stop you from changing the defaults to a higher level but if you’re looking to display thousands of records on a webpage without paging through your data then this is seen as poor design, here is a nice a quote from the RavenDB website: –

RavenDB will let you shoot yourself in the foot, but only after you make it absolutely clear that this is what you actually want to do

It seems to be quite common for developers to try to work around the limits and actually want to block their leg off, this usually stems from a misunderstanding of how to use RavenDB, it really is a very simple concept which I find to be a nice feature.

RavenDB also had a very useful option to stop yourself from creating the dreaded SELECT N+1 scenario – this feature stops after 30 request to the server per session, again configurable but if you’re hitting this limit then your more than likely querying incorrectly.

To summarise safe by default is there to help and not hinder, yes you’ll write code then realise your only returning 128 records when there should be more results, no doubt everyone has done that whilst using RavenDB, either your unit tests aren’t right or you’ve forgotten to use .Take()


Tags:


Free RavenDB Talk in Glasgow on November 14th

My colleague at work Mohammed Ibrahim is doing a free talk titled RavenDB: working with NoSQL data in .NET at Glasgow Caledonian University on Nov 14.

If your unfamiliar with noSQL and want to know more then come along and you will learn all about it and more.

So come along, learn, ask some questions and to get an insight into why RavenDB rocks, register at http://ravendbglasgow.eventbrite.com – tickets are selling fast so get yours while you can.

There will be free swag! to give away at the event, including t-shirts and RavenDB stickers.



What I learned last week – 3rd September – 7th September

This past week I have been mainly working with KnockOutJS, if you looking for a way to create a rich UI that users will love and your building a website then I highly recommend you take a look at KnockoutJS

What I learned this past week

  • Web Essentials 1.1 for VS 2012 by Mads Kristensen has been updated, you can read more here – go check it out.
  • RavenDB Backups – I decided to instead of using the RavenDB Backup.exe – I am going to stick with just exporting the tenant database and saving this off to a network share.
  • At work we are looking at introducing CoffeeScript – CoffeeScript is a language that sits on top of JavaScript and compiles down to JavaScript but has added benefits including making your code cleaner and has much more signal to noise ratio – more on CoffeeScript soon , I’m off to learn more on it via PluralSight.
  • Redgate have taken on the 2 developers who write Glimpse and will be adding to the project – you can read more here.

What did you find useful this week? – please let me know by leaving a comment after the beep.



RavenDB – using the uniqueconstraint bundle

During the week at work we came across an issue where we are doing a linq query which is using a statement such as this:-
[sourcecode language=”csharp”] var product = session.Query().Single(GetSource(key));
[/sourcecode]

In Sql Server land we have a product table and the field productcode is unique – we noticed that in RavenDB we were seeing products
which had the same productcode in different documents (not unique), this was not the desired outcome and so we looked into how to go about fixing this in RavenDB, enter the uniqueconstraints RavenDB bundle.

RavendDB bundles are like add-ons and Raven comes with quite a few and they can be exceptionally handy. To use the uniqueconstraints bundle you create a folder and call it plugins, usually you keep this inside the server folder but you can put it anywhere and add a config value so that Raven can find your plugins, an example of how to set up your config to locate plugins:-

[sourcecode language=”xml”] <?xml version="1.0" encoding="utf-8" ?>
<configuration>
<appSettings>
<add key="Raven/Port" value="*"/>
<add key="Raven/DataDir" value="c:\RavenDB\Data"/>
<add key="Raven/IndexStoragePath" value="c:\RavenDB\Data\Indexes" />
<add key="Raven/Esent/LogsPath" value="c:\RavenDB\Data\Logs" />
<add key="Raven/PluginsDirectory" value="c:\RavenDB\Plugins"/>
<add key="Raven/AnonymousAccess" value="All"/>
<add key="Raven/ResetIndexOnUncleanShutdown" value="true"/>
</appSettings>
<runtime>
<loadFromRemoteSources enabled="true"/>
<assemblyBinding xmlns="urn:schemas-microsoft-com:asm.v1">
<probing privatePath="Analyzers"/>
</assemblyBinding>
</runtime>
</configuration>
[/sourcecode]

Server side checking

  • To to use the uniqeconstraints bundle add the Raven.Bundles.UniqueConstraints.dll from the bundles folder to your plugins folder where ever you decide to put them.
  • Then restart RavenDB
  • To see if they have been installed and picked up – go to the Raven Management Studio and then at the bottom left hand side, click on statistics – next to where it says Triggers it should say something like Raven.Bundles.UniqueConstraints.UniqueConstraintsDeleteTrigger

Client Side checking

In order to implement the check through code we borrowed the example written by Richard Dingwall taken from his blog.

[sourcecode language=”csharp”] namespace WhateverNameSpaceYouHave.Extensions.RavenUniqueConstraint
{
using System;
using System.Linq.Expressions;

using Raven.Client;

public interface IRavenUniqueInserter
{
void StoreUnique(IDocumentSession session, T entity, Expression<Func<T, UTunique>>; keyProperty);
}
}
[/sourcecode]

Aboove the interface, below that code that uses it:-

[sourcecode language=”csharp”] namespace WhateverNameSpaceYouHave.Extensions.RavenUniqueConstraint
{
using System;
using System.Linq.Expressions;

using Raven.Abstractions.Exceptions;
using Raven.Client;

public class RavenUniqueInserter : IRavenUniqueInserter
{
public void StoreUnique(IDocumentSession session, T entity, Expression<Func<T, Tunique>> keyProperty)
{
if (session == null)
{
throw new ArgumentNullException("session");
}

if (keyProperty == null)
{
throw new ArgumentNullException("keyProperty");
}

if (entity == null)
{
throw new ArgumentNullException("entity");
}

var key = keyProperty.Compile().Invoke(entity).ToString();

var constraint = new UniqueConstraint { Type = typeof(T).Name, Key = key };

DoStore(session, entity, constraint);
}

private static void DoStore(IDocumentSession session, T entity, UniqueConstraint constraint)
{
var previousSetting = session.Advanced.UseOptimisticConcurrency;

try
{
session.Advanced.UseOptimisticConcurrency = true;
session.Store(constraint, string.Format("UniqueConstraints/{0}/{1}", constraint.Type, constraint.Key));
session.Store(entity);
session.SaveChanges();
}
catch (ConcurrencyException)
{
// rollback changes so we can keep using the session
session.Advanced.Evict(entity);
session.Advanced.Evict(constraint);
throw;
}
finally
{
session.Advanced.UseOptimisticConcurrency = previousSetting;
}
}
}
}
[/sourcecode]

How to call it from code
[sourcecode language=”csharp”] try
{
new RavenUniqueInserter().StoreUnique(session, destination, x =&gt; x.Code);
LogManager.GetCurrent().Event(LoggingLevel.Info, "Inserting Product:+" + destination.Code, "Method name here");
}
catch (ConcurrencyException)
{
LogManager.GetCurrent().Event(
LoggingLevel.Error,
"Product code already in use. Code: " + destination.Code, "Method name here");
}
[/sourcecode]

And thats it, now we wont be able to add a product with a productcode which already exists in our RavenDB database, making it a unique constraint just as youd have in Sql.

Please add a comment or ask a question if you found this useful.




What I learned last week – July 23-27th

This is the first in a series I am going to do on what I learned last week – it will cover what I’ve been up to and what I have read and learnt throughout the week – ok let’s get started.

Found a really nice sample on how to notify users when an action on your site has taken place – say you save a record or delete a record, perhaps you need to show a validation message, maybe even a banner at the top that has to stay there,

Take a look at the oh so simple to use JQuery plugin called Notify.

Awesome Stuff

  • Visual Studio 2012 is really nice to use! – you should take a look now if you havent already.
  • Shout out to the guys at RavenDb highly recommend it, also note its FREE until you need to release to production, so if you’re doing demos, needing a quick win on a project – take it for a spin.
  • Resharper 7 is out – if you’re not already using this then do yourself a favour, why walk when you can run, go try it now Resharper 7.
  • Redgate have brought out a plugin for setting up a database build in TeamCity Redgate TeamCity Plugin

Please leave a comment after the beep.



AspConf – My Review

Last week we had AspConf – “aspConf is a virtual conference focused on one thing: showcasing the ASP.NET stack as the platform for great web applications.

Over 2 days they had 5 virtual rooms used to present lots of fascinating stuff on .Net, including talks about all manner of things in the Asp.Net world.

The sessions on both days covered topics such as, Async in .Net 4.5, Glimpse, TDD, Azure and Cloud Computing and lots lots more, you can read more about the sessions here.

Although it had some technical difficulties, due to its popularity I may add, the conference is an awesome way to learn for free – and free is always good. It’s a superb way to learn about content you havent had time to look at it, or new technologies you havent heard of. Hopefully next year they might try to use something different from livemeeting.

If you missed any of the session then fret not, most of them if not all of them were recorded and put up on Channel 9 here

I only managed to see sessions from day 2 so I am off to go watch more content on Channel 9 – a great resource for free learning.



RavenDB CRUD – How To

I always wondered how easy it would be to do a RavenDB CRUD (Create, Read, Update, Delete) example, so lets create a simple MVC application which we can use to create, read, update and delete.

The following is some very basic code and yes you guessed it not for production but more of a quick sample to get you on your way with RavenDB and adding, editing and deleting documents.

We shall use a product model as our basis for the demo application – let’s get started.

  1. Create a blank MVC solution to begin with
  2. Using Nuget add in the Nuget package called RavenDB
  3. Lets setup RavenDB as we require within the global.asax.cs file as below:-
    [sourcecode language=”csharp”] protected void Application_Start()
    {
    AreaRegistration.RegisterAllAreas();

    RegisterGlobalFilters(GlobalFilters.Filters);
    RegisterRoutes(RouteTable.Routes);

    Store = new DocumentStore { ConnectionStringName = "RavenDB", DefaultDatabase = "Products" };
    Store.Initialize();
    }
    [/sourcecode] Note:- I have added in a DefaultDatabase config value above which means I have created a new Products Raven Database within the RavenDB Management Studio called Products, if you don’t do this and just want to use the default database then remove the part DefaultDatabase = “Products”

  4. Add a class into the Models folder called Product as below:-
    [sourcecode language=”csharp”] public class Product
    {
    public string Id { get; set; }
    public string CategoryId { get; set; }
    public string SupplierId { get; set; }
    public string Name { get; set; }
    public string Code { get; set; }
    public decimal StandardCost { get; set; }
    public decimal ListPrice { get; set; }
    public int UnitsOnStock { get; set; }
    public int UnitsOnOrder { get; set; }
    public bool Discontinued { get; set; }
    }
    [/sourcecode]
  5. Now that we have our Model, let’s go ahead and add in a couple of controllers. We will use a base controller when working with our RavenDB project so firstly create a RavenDB Controller, to do this right-click on controllers folder and then select Add then choose Controller, select Empty Controller.
  6. Paste this code into your RavenController:-
    [sourcecode language=”csharp”] public class RavenController : Controller
    {
    public IDocumentSession _session { get; private set; }

    public const int DefaultPage = 1;
    public const int PageSize = 10;

    protected override void OnActionExecuting(ActionExecutingContext filterContext)
    {
    if (filterContext.IsChildAction)
    return;
    _session = MvcApplication.Store.OpenSession();
    base.OnActionExecuting(filterContext);
    }

    protected override void OnActionExecuted(ActionExecutedContext filterContext)
    {
    if (filterContext.IsChildAction)
    return;

    using (_session)
    {
    if (filterContext.Exception != null)
    return;

    if (_session != null)
    _session.SaveChanges();
    }
    }

    protected int CurrentPage
    {
    get
    {
    var s = Request.QueryString["page"];
    int result;
    if (int.TryParse(s, out result))
    return Math.Max(DefaultPage, result);
    return DefaultPage;
    }
    }
    }
    [/sourcecode]

  7. Lets now add a ProductController and this time choose, Controller with empty read/write actions as below:-
    ProductController
  8. We will start off by adding in some code which I have already written for the methods which we will use on the Products Controller, add them into your code:-
    [sourcecode language=”csharp”] public class ProductController : RavenController
    {
    //
    // GET: /Product/

    public ActionResult Index()
    {
    var model = _session.Query<Product>()
    .Paging(CurrentPage, DefaultPage, PageSize)
    .ToList();

    Mapper.Map<List<Product>, List<ProductViewModel>>(model);

    return View(model);
    }

    //
    // GET: /Product/Details/5

    public ActionResult Details(string id)
    {
    var model = _session.Load<Product>(id);
    return View(model);
    }

    //
    // GET: /Product/Create

    public ActionResult Create()
    {
    var model = new Product();
    return View(model);
    }

    //
    // POST: /Product/Create

    [HttpPost] public ActionResult Create(Product product)
    {
    try
    {
    _session.Store(product);
    return RedirectToAction("Index");
    }
    catch
    {
    return View();
    }
    }

    //
    // GET: /Product/Edit/5

    public ActionResult Edit(string id)
    {
    var model = _session.Load<Product>(id);
    return View(model);
    }

    //
    // POST: /Product/Edit/5

    [HttpPost] public ActionResult Edit(Product product)
    {
    try
    {
    _session.Store(product);
    return RedirectToAction("Index");
    }
    catch
    {
    return View();
    }
    }

    //
    // GET: /Product/Delete/5

    public ActionResult Delete(string id)
    {
    var model = _session.Load<Product>(id);
    return View(model);
    }

    //
    // POST: /Product/Delete/5

    [HttpPost, ActionName("Delete")] public ActionResult DeleteConfirmed(string id)
    {
    try
    {
    _session.Advanced.DatabaseCommands.Delete(id, null);
    return RedirectToAction("Index");
    }
    catch
    {
    return View();
    }
    }

    public ActionResult StoreSomeProductInDatabase()
    {
    var product = new Product
    {
    Name = "Product Name",
    CategoryId = "category/1024",
    SupplierId = "supplier/16",
    Code = "H11050",
    StandardCost = 250,
    ListPrice = 189
    };

    _session.Store(product);
    _session.SaveChanges();

    //return Content(product.Id);
    return RedirectToAction("Index");
    }

    public ActionResult InsertSomeMoreProducts()
    {
    for (int i = 0; i < 50; i++)
    {
    var product = new Product
    {
    Name = "Product Name " + i,
    CategoryId = "category/1024",
    SupplierId = "supplier/16",
    Code = "H11050" + i,
    StandardCost = 250 + (i*10),
    ListPrice = 189 + (i*10),
    };
    _session.Store(product);
    }

    _session.SaveChanges();

    //return Content("Products successfully created");
    return RedirectToAction("Index");
    }

    public ActionResult GetProduct(int id)
    {
    Product product = _session.Load<Product>(id);
    return Content(product.Name);
    }

    public ActionResult LoadAndUpdateProduct()
    {
    Product product = _session.Load<Product>("products/5");
    product.ListPrice -= 10;
    _session.SaveChanges();
    return Content("Product 5 successfully updated");
    }

    public ActionResult DeleteProduct(int id)
    {
    Product product = _session.Load<Product>(id);
    if (product == null)
    return HttpNotFound("Product {0} does not exist");
    _session.Delete(product);
    _session.SaveChanges();
    return Content(string.Format("Product {0} successfully deleted", id));
    }

    /// <summary>
    /// Get all the products that are available for sale (discontinued equal to false) ordered by the product’s list price
    /// </summary>
    /// <returns></returns>
    public ActionResult GetDiscontinuedProducts()
    {
    var products = from product in _session.Query<Product>()
    where product.Discontinued == false
    orderby product.ListPrice
    select product;

    return View(products.ToList());
    }
    }
    [/sourcecode]

  9. Ok so above we added a lot of code but this is good as this is the entire product controller and will allow us to add/edit/delete new products into RavenDB.
  10. I also added in a couple of methods for creating some test data, and also a method for returning discontinued products, mainly to show how you can go about doing this.
  11. Some screen shots are below, note they show me using RavenDB Profiler and MiniProfiler:-

    Crud 1

    Crud 2

    Crud 3

The code for this article is up on github here.
For more on RavenDB have a look at my other RavenDB posts here.

Please leave a comment, ask a question, find a bug or anything like that please let me know.




Exporting and Importing data with RavenDB

I was wondering how I would go above upgrading from one build version of RavenDB (build 701) to a newer version of RavenDB (build 888) and decided to find out how.

As it turns out like most things in RavenDB its very easy! – the following is how to go about upgrading from one version of RavenDB to another.

Note:- You can do it this way and this is the recomended way here however I wanted to try doing it using importing and exporting the data.

Please not that this was done on my local dev machine at work and not done in a production environment, it wont cover replication as well as some other things to think about but its covers the basics.

I have RavenDB build 701 running from d:\ravedb-build\701 which runs on http://localhost:8080, I then added some test documents to the default database.

I noticed RavenDB had been updated to Build 888 and I began to think how do I get the data out of the old build version and into the new version, having a quick look I soon realised that I need to export data from the 701 build and then import that data into the 888 build.

Here is how I went about it:-

  1. I have RavenDB build 701 running from c:\ravendb\RavenDB-Build-701\ and had 50 documents within it – this runs from http://localhost:8080 as seen below:- Raven Build 701
  2. I then downloaded RavenDB build 888 and ran that from d:\RavenDB-Build-888\ which was empty – this runs from http://localhost:8082 as seen below:- Raven Build 888
  3. Now we can see the differences in the document database as below, Raven 701 build with documents:-
    701 Studio
  4. And the Raven 888 build with no documents as yet:-
    888 Studio
  5. To export and import data within RavenDB you use a tool called smuggler, smuggler can be found within the Smuggler folder and the exe is called Raven.Smuggler.exe
  6. To export the data from RavenDB Build 700 I went into a command prompt window moved into the location of the smuggler exe and ran the following command -> Raven.Smuggler out http://localhost:8080 dump.raven
    Dump of 701
  7. Ok great, now we have exported our data, time to import it into the newer 888 build of RavenDB, to do this we need to a couple of things, firstly locate the dump.raven file in the 701 build/smuggler folder as below:-

    701 dump file location

    And then copy it to somewhere for importing, to make it easy I just copied it to the smuggler folder within the build 888 as below:-

    888 dump file location

    Then go to the smuggler folder for this build and then use the following command -> Raven.Smuggler in http://localhost:8082 dump.raven

  8. We will see Raven ha imported our documents for us:- Data Imported
  9. Lastly lets check in the new build 888 that our documents are there:-
    Documents Imported

And there we go we have moved out documetns from the 701 build to the 888 build, if yo do this you will notice that the index definitions are copied there but havent been built, i.e. youll need to rubuild them manually.

Please feel free to leave a comment if you find this useful or have any questions.