<p>Could someone explain? I understand the basic concepts behind them but I often see them used interchangeably and I get confused. </p>

<p>And now that we're here, how do they differ from a regular function?</p>

Not all closures are lambdas and not all lambdas are closures. Both are functions, but not necessarily in the manner we're used to knowing.
A lambda is essentially a function that is defined inline rather than the standard method of declaring functions. Lambdas can frequently be passed around as objects.
A closure is a function that encloses its surrounding state by referencing fields external to its body. The enclosed state remains across invocations of the closure.
In an object-oriented language, closures are normally provided through objects. However, some OO languages (e.g. C#) implement special functionality that is closer to the definition of closures provided by purely
functional languages
(such as lisp) that do not have objects to enclose state.
What's interesting is that the introduction of Lambdas and Closures in C# brings functional programming closer to mainstream usage.