var list = new CouchbaseList<dynamic>(bucket, "myList");I blogged last year about my switch from Markdown to AsciiDoc, and that I was using AsciiDocFX.
I still like AsciiDoc, but AsciiDocFX has been getting on my nerves:
- It doesn't have a very good update system. It checks for new versions, but it seems like I have to a) uninstall the old version, b) reinstall the new version, otherwise I get problems. I may be doing it wrong, but this made me want to update less.
- The UI is a little wonky. The live preview sometimes seem to keep its update a few keystrokes behind, meaning that the preview and the document are out of sync. It also tends to get locked up, until I click the Restore button and then Maximize button.
Still, I used it.
But, I built a new computer this week. And I've been setting up my softwares on it. I thought it might be worthwhile to see if there's an AsciiDoc plugin for Visual Studio Code. And, of course there was, because apparently the Code extension ecosystem is booming!
So, I installed AsciiDoc by Joao Pinto, since it came with a live preview. But, it requires me to actually install the asciidoc command line tool.
See?
So, I thought, that should be easy enough. I went to the AsciiDoc site and started following the directions for Windows installation.
Install Ruby
Okay, well now I need to install Ruby. Should be easy enough. I already have Chocolatey NuGet, so I'll just run choco install ruby. No problem. I know that ruby comes with gem, so I should be all set.
Install AsciiDoc
According to AsciiDoc... docs... I just use gem install asciidoctor and that should do the trick.
But, no. It's not that easy. Otherwise I wouldn't be writing this blog. I got an error message:
SSL_connect returned=1 errno=0 state=SSLv3 read server certificate B: certificate verify failed
I'm sure all you Ruby people or Mac people or whatever already know where this is going, but I had no clue. So I googled it. I found a whole bunch of suggestions on StackOverflow. Some solutions made sense but weren't for Windows, and vice versa. I eventually hit upon some random guy's Gist and SSL upgrades on rubygems.org which lead to me this page on rubygems.org about SSL updates.
Fixing RubyGems Certificate Thingy
So, following that literally:
gem install --local C:\rubygems-update-2.6.7.gem
and then
update_rubygems --no-ri --no-rdoc
and finally
gem uninstall rubygems-update -x
So, I guess that fixed... something? It's described in the gist I linked above. But I don't really understand why it's still a problem for a brand new install of ruby. Not complaining! It worked!
Okay, now Install AsciiDoc
So now gem install asciidoctor works. And now I get a live preview of AsciiDoc in Visual Studio Code.
I'll report back after some more time blogging to see if I like this, or if I eventually go back to AsciiDocFx.
UPDATE: As of early April 2017, I've been blogging this way and I'm extremely happy with it. Visual Studio Code keeps getting better, the preview plugin keeps getting better. I've had to introduce a few tweaks in my process, but I've got the whole pipeline semi-automated. It's easy to bring in code samples (thanks to AsciiDoc), easy to post to both the Couchbase blog and this blog, easy to run Yoast on it, and so on. Some day I'll write a blog post explaining the whole process (and hopefully get some good suggestions for streamlining!)
This is a repost that originally appeared on the Couchbase Blog: .NET Core List, Queue, and Dictionary Data Structures backed by Couchbase.
The addition of the sub-document API to Couchbase 4.5 has paved the way for efficient data structure support in Couchbase.
In this blog post, I’m going to show a demo of three types of data structures you can use with the Couchbase .NET SDK:
-
List - a list of objects, basically a List<T> backed by Couchbase
-
Queue - a queue of objects, basically a Queue<T> backed by Couchbase
-
Dictionary - a dictionary of objects, basically a Dictionary<K,T> backed by Couchbase
I’ll also discuss a little bit how this works behind the scenes.
You can play along at home if you like. The source code for this blog is available on GitHub, and Couchbase Server is free to download (developer previews of version 5 are currently available monthly).
List
A List<T> is a .NET data structure that is held in memory. With the data structures provided by the Couchbase .NET SDK, you can store it in a Couchbase document.
To create a Couchbase-backed List:
The string "myList" corresponds to the key for the document that will contain the list. When using CouchbaseList<T>, a single document with that key will be created (if one doesn’t exist already). If a document by that key already exists, CouchbaseList will use it.
You can now add/remove items from the list and that will all be persisted to the document. You can also perform other operations like getting a count of the items in the list.
// add 10 objects to the list
for(var i = 0; i < 10; i++)
list.Add(new { num = i, foo = "bar" + Guid.NewGuid()});
// remove an item from the list by index
list.RemoveAt(5);
// show an item from the list by index
Console.WriteLine("5th item in the list: " + list[5].foo + " / " + list[5].num);The above code would result in a document with a key "myList" that looks like below. Notice that the item with num of 5 is not listed, because it was removed.
There’s something subtle in the above example that needs to be pointed out. Notice that I used var item = list[5]; and then item.foo and item.num in the WriteLine. If I used list[5].foo and list[5].num directly, that would result in two different subdocument calls to Couchbase. Not only is this less than optimal efficiency, but it’s possible for the values to change between the two calls.
[
{
"num": 0,
"foo": "bara1fd74ee-a790-4a0f-843c-abe449cb8b1d"
},
{
"num": 1,
"foo": "bardc1d8f9a-4e93-46f9-b8ae-ec036743869e"
},
{
"num": 2,
"foo": "bar9a60abe9-1e04-4fba-bd1f-f1ec39d69f56"
},
{
"num": 3,
"foo": "bar9566605b-7abf-4a0c-aa9d-63b98ce86274"
},
{
"num": 4,
"foo": "bar6261323f-de50-42a7-a8a7-6fcafb356deb"
},
{
"num": 6,
"foo": "bar13832bcb-2aa0-491a-a01f-1d496f999ffc"
},
// ... etc ...
]Queue
Very similar to List, you can create a Couchbase-backed queue:
var queue = new CouchbaseQueue<dynamic>(bucket, "myQueue");A queue is stored just like a list. The difference is that the ordering is significant, and this is reflected by the operations you perform on a queue: Enqueue and Dequeue.
for(var i = 0; i < 3; i++)
queue.Enqueue(new { num = i, foo = "baz" + Guid.NewGuid()});
// dequeue
var item = queue.Dequeue();
Console.WriteLine("item num " + item.num + " was dequeued. There are now " + queue.Count + " items left in the queue.");The above code would result in a document with a key "myQueue" (see JSON below). Notice there is no object in the array with num "0" because it was dequeued.
[
{
"num": 1,
"foo": "baz64bb62b6-bf23-4e52-b584-d2fa02accce6"
},
{
"num": 2,
"foo": "baz0a160bd9-aa7b-4c45-9e19-d1a3d982a554"
}
]Dictionary
Hopefully you’re seeing a pattern now. To create a dictionary:
var dict = new CouchbaseDictionary<string,dynamic>(bucket, "myDict");Again, a document will be created with the given key. The operations that can be performed include Add, Remove, and the indexer [] operation.
for(var i = 0; i < 5; i++)
dict.Add("key" + Guid.NewGuid(), new { num = i, foo = "qux" + Guid.NewGuid()} );
// print out keys in the dictionary
Console.WriteLine("There are " + dict.Keys.Count + " keys in the dictionary.");
foreach(var key in dict.Keys)
Console.WriteLine("key: " + key + ", value.num: " + dict[key].num);A dictionary document looks like:
{
"key5aa2520d-123c-4fca-b444-b0cb8846d46e": {
"num": 0,
"foo": "qux93b197dc-f175-4246-a38d-7b080eb9bea0"
},
"key55dee298-14c6-4da7-97a8-66c69d7e8a70": {
"num": 1,
"foo": "quxa593ee4c-682c-402d-887b-3f09f029e9b6"
},
"key3386afcf-7b70-4e4d-b9ae-6defbca33fe7": {
"num": 2,
"foo": "qux1259ae94-1008-4e1f-86a1-bfbd0873b09b"
},
"key2bc8c451-f125-4282-9fb4-7ea15f4b3168": {
"num": 3,
"foo": "qux1b6fb62b-9918-46dc-9a2f-610a55d017ef"
},
"key3f7041f3-abd3-49c7-a373-454cbd2ac0fc": {
"num": 4,
"foo": "qux0a87655f-197d-4fb2-8a54-b1de6e288de4"
}
}A note about C# dynamic: I used dynamic to keep the code samples short and simple. In your application, you are probably better off using a real defined C# type. It all gets serialized to JSON in Couchbase, of course.
Behind the scenes
Before the subdocument API was released in Couchbase Server 4.5, these data structures were possible, of course. The catch was that you would be loading up the entire document, putting it in a list, making changes to the list, and then saving the entire document. If you have large data structures, but are only reading or making changes to a single item, this would often result in wasted time and wasted bandwidth and possibly increased contention.
The subdocument-API (which you can use directly; I covered it in the Sub-document API in Couchbase Server 4.5 with the .NET SDK (revisited) blog post) is used behind the scenes in CouchbaseList, CouchbaseQueue, and CouchbaseDictionary. So when you add an item to a CouchbaseList, for instance, only that item is being sent over the wire, not the entire list.
Some operations will still need to get the entire document. For instance, iterating through a collection using a foreach loop will retrieve the full document. Removing an item from a list will result in the full document being scanned. But if sub-document operations come along in the future to support those actions, the SDK implementations will be updated accordinging.
Summary
These data structures are another tool to help you manage your data. Since they use the sub-document API, they are generally more performant than a whole-document approach. For more detail, check out the Data Structures documentation.
Have questions? Feedback? Need help? Please visit our forums, ping me on Twitter @mgroves, or leave a comment.
This is a repost that originally appeared on the Couchbase Blog: A tour of the new Couchbase Web Console (video).
Earlier this month, we introduced Developer Builds. In this video, I take you on a quick tour of the new Couchbase Web Console.
This is a video version of the blog I wrote earlier about the new Couchbase Web Console.
If you have questions or feedback, please contact me at matthew.groves@couchbase.com, or on @mgroves at Twitter, or just leave a comment below.
This is a repost that originally appeared on the Couchbase Blog: A tour of the new Couchbase Web Console.
Change is coming to the Couchbase Web Console that you know and love. But don’t worry, the old UI will still be available (for a while).
In this blog post, I’m going to take you on a tour of the new UI. I’ll be pointing out some of the differences and similarities. If you have anxiety about change, don’t worry: the purpose of this change is to improve usability, performance, and efficiency. The new UI gives Couchbase Web Console room to grow with new features that are coming down the road.
An important note!
This blog and the screenshots in it were taken from an early developer build. There may be other changes and improvements that show up by the time the release is in your hands, so the final product might be different. Feel free to point out any differences in the comments below.
Installation and Setup
The installation and setup process has not changed much, but it does get a fresh coat of paint. The installer itself appears the same (although the setup wizard is likely to be revised in future builds). Once the installer is complete, you’ll get redirected to a browser (if you’re installing Couchbase Server locally, this means a browser pointed to http://localhost:8091 will appear). Functionally, this part of the experience is the same as before, but it gets a new coat of paint.
Figure 1. The setup screen forsCouchbase Server 5.0
After you click the "Setup" button, you’ll be taken through the installation wizard. Depending on which version you have been using, there may be some new options here that you haven’t seen. I’m not going to cover them in this post, but stay tuned for more blog posts about upcoming features in the latest releases.
Figure 2. Starting a new cluster in Couchbase Server setup
The sample buckets are still available. Note that these buckets (as always) are not password protected, and are meant for development only. Please check out the recent Advisory Note about Couchbase security.
Figure 3. Loading a sample bucket (optional) during Couchbase Server setup
You can also choose to create a bucket named "default" during setup. Again, this is not recommended for production (even if you put a password on it), since it’s a well-known bucket name that makes it a target. It’s also optional, you can click "Skip".
Figure 4. Create a default bucket (optional) during Couchbase Server setup
Notification and Terms and Conditions are next. We are offering the full Enterprise edition in the developer build, so you can check out the Enterprise features.
Figure 5. Accept terms and conditions of the Enterprise Edition and sign up for notifications
Finally, create an admin user. Once again, I recommend not using "Administrator" as the user name in production, as it is a well-known default. Also, make sure to use a strong password.
Figure 6. Create an administrator user for Couchbase Server
Using the new Couchbase Web Console
Once you’ve gone through the familiar install process, it’s time to see some of the substantive changes to the UI.
The first thing you’ll see is the dashboard screen, which has been streamlined.
Figure 7. The new look of the Couchbase Server Web Console
Some things that jump out at me immediately:
The design. The new Couchbase Server UI is heavily influenced by Google’s Material Design. You will see more evidence of this as you navigate around the site. Material Design is meant to provide a "design language" that is geared toward "digital material" that can expand/change intelligently.
Main Navigation. The navigation is now vertical and on the left side of the page. Also notice that is is organized by "cluster" and "data".
Figure 8. Main navigation is now vertical
Servers
Click on the "Servers" link to see the new look of the servers page.
Figure 9. The Servers page shows a list of nodes
This should look relatively familiar. Each node is listed here, with a list of services running on each node and system metrics. I only have one node in my example, but each node would be listed here in a production deployment. Notice that there are filters "Active Server" and "Pending Rebalance" that have a rounded highlight. Next to them are some buttons to manage servers and server groups which use rectangle shapes. These shapes help to provide visual cues no matter what the size of your screen is.
Security
Continuing the tour, click on the "Security" navigation link.
Figure 10. Couchbase Server Security UI
Not much different on this screen compared to the old UI, but I do want to draw your attention to the sub-navagation at the top: "Internal User/Roles", "Root Certificate", and "Audit". The secondary navigation across the site will be across the top of the page like this.
There are some new features coming soon for role-based security. I would expect the UI here to evolve in future versions to accomodate.
Query Workbench
As a developer, this is always my favorite part of the Couchbase Console, and the one I spend most of my time in. Besides the obvious design change, there are some minor improvements and changes here that I’d like to call attention to.
Figure 11. New design of the Query Workbench
No more Clear button. I use the clear button a lot, but my usage is probably atypical from a typical Couchbase user. The clear button is a frustrating button to hit accidentally while working on queries, so it’s been removed.
History. Click on "history" to see a list of all the queries that you’ve run. It’s searchable! Which can be very useful after a long day of query writing and data modeling. Remember that "clear" button? It’s here in the history popup; you can still access it, but you’re less likely to click it by accident.
Figure 12. Query Workbench history with search
Data Bucket Insights. These have been moved to the right side of the screen. Personally, I think this is a better use of space. (This is an Enterprise Edition feature).
Plan and Plan Text options in results. By default, the queries you run will be EXPLAINed, and you can view the output of that in the Plan and Plan Text results. To turn that feature off, click the settings icon next to "Query Editor".
Figure 13. Plan and Plan Text results tabs
Query Monitor. Notice the "Query Monitor" secondary navigation link at the top. This page shows you information about some of the system catalogs that were added in Couchbase Server 4.5.
Buckets
As a developer, I spend a lot of time in the Buckets section of the UI. The new version of this page isn’t wildly different, but I have found it a little snappier.
Figure 14. New Bucket management UI
Click on a row to expand information about the bucket (as well as to expose the Delete/Compact/Edit buttons).
Statistics and Charts
One significant area of change is the way that charts and statsitics are shown in the UI. Clicking a bucket name will still bring you to the analytics and information that you’re used to.
Figure 15. Chart that shows the number of N1QL requests
There aren’t any new charts that I’m aware of, but there have been changes made to improve usabilty of the charts as well as get them to adhere closer to the Material Design philosophy. They also take up the full space of the page, to maximize your view.
Figure 16. Chart that shows the number of HTTP requests
It’s difficult to demonstrate these changes in a blog post with static images (I may make a video walkthrough in the future), so I definitely recommend that you install the latest version of Couchbase Server 5.x and give them a try.
Why did you move my cheese!
Hopefully, these UI changes delight you and make your job easier. However, abrupt change can be difficult. This is why there is still an option to view the "Classic UI". Just click this link (at the top right of the page) and you’ll be albe to view the old UI. I don’t know how long the old UI is going to stick around.
Figure 17. Use this link to show the old "Classic UI"
To switch back to the new UI from the old UI, just click "New UI".
Figure 18. Use this link to go back to the "New UI"
It’s likely that new features will only appear in the new UI.
Feedback and Summary
Stay tuned to the Couchbase Blog for information about what’s coming in the new UI in the next developer build.
I’ve given you the basic tour, but to really get a feel for the UI, I recommend you download and try it. Download Couchbase Server 5.0 today!
We want feedback! Developer releases are coming every month, so you have a chance to make a difference in what we are building.
Bugs: If you find a bug (something that is broken or doesn’t work how you’d expect), please file an issue in our JIRA system at issues.couchbase.com. Or, contact me with a description of the issue. I would be happy to help you or submit the bug for you (my Couchbase handlers give me a candy bar and pat me on the head every time I submit a good bug).
Feedback: Let me know what you think. Something you don’t like? Something you really like? Something missing?
In some cases, it may be tricky to decide if your feedback is a bug or a suggestion. Use your best judgement, or again, feel free to contact me for help. I want to hear from you. No suggestion is too small! The only stupid question is the one you don’t ask! The best way to contact me is either Twitter @mgroves or email me matthew.groves@couchbase.com.
My job as a developer evangelist for Couchbase means than I work much less with SQL Server than I used to. However, it doesn't mean I don't keep up with it.
In fact, inspired by Couchbase, I decided to try out using SQL Server as a sort of document data store. SQL Server 2016 introduced some interesting new features that make this kinda possible: JSON_VALUE, JSON_QUERY, and JSON_MODIFY.
I set up a "document" table, which is two fields: a guid and an nvarchar(max). This is kinda like a Couchbase bucket: a key and a JSON document to go with it.
I put fairly complex hierarchical documents in these fields, something like:
I say "fairly complex", because representing this in a standard normalized fashion would require at least two tables, foreign keys, constraints, and then data migration and schema migration as the model evolves.
Now, suppose I want to execute a query and find all the document that are "Show=true". With Couchbase and N1QL, I would just use something like "SELECT * FROM `bucket` WHERE show = true".
With SQL Server, there's a little more work. "SELECT t.key, t.doc FROM [table] t WHERE JSON_VALUE(t.SpeakingInfo,'$.Show') = 'true'". Notice that JSON_VALUE is being applied to a text field, and a JSON path is used within JSON_VALUE to get a specific value from within that JSON object.
For this simple project I'm doing, that's all I need. No idea yet what kind of performance level I can expect from JSON_VALUE and more complex JSON paths.
But, this is definitely an example of the kinds of database convergence I've been telling people about. The separations between NoSQL and SQL are becoming less strict, at least in terms of data modeling and querying.
You need to be using SQL Server 2016 or SQL Server Azure to take advantage of the JSON_* functions.
