Attendees: beyonddc, clsk, fred, J_K9

Note: All timestamps in UTC+2 (Central European Summer Time).

---

Oct 23 23:22:15 <J_K9> hello clsk, fredm, beyonddc ;)

Oct 23 23:22:26 <clsk> hi

Oct 23 23:22:44 <fredm> hey!

Oct 23 23:22:48 <J_K9> it's a better turnout than I had anticipated! great! :)

Oct 23 23:23:12 <J_K9> we'll wait until 21:30 just in case anyone else joins

Oct 23 23:28:25 <beyonddc> brb in couple minutes.

Oct 23 23:29:06 <J_K9> ok - we'll start when you get back :)

Oct 23 23:34:24 <beyonddc> back

Oct 23 23:34:31 <J_K9> ok, great

Oct 23 23:34:49 <J_K9> the first module I want to talk about is the Storage module

Oct 23 23:35:08 <J_K9> I want us to try to think of its requirements in order of implementation/importance

Oct 23 23:36:04 <J_K9> so, let's shoot:

Oct 23 23:36:20 <J_K9> what backends are we going to allow Utilities to store data in? Just database and filesystem?

Oct 23 23:37:09 <beyonddc> Is it suppose to be version controlled also?

Oct 23 23:37:20 <J_K9> ah, yes, that's right

Oct 23 23:37:35 <beyonddc> I think myself and Fred are still at work so we might be a little slow in response.

Oct 23 23:37:38 <J_K9> how about XML files instead of database? what do you think of that?

Oct 23 23:37:42 <J_K9> no problem

Oct 23 23:38:01 <J_K9> it would definitely simplify the implementational requirements on the client end

Oct 23 23:38:13 <fredm> backend - as in svn? I'm looking at the layers diagram

Oct 23 23:38:41 <beyonddc> Yes Fred, our thought was to use SVN as the backend to provide version control.

Oct 23 23:39:02 <J_K9> yes - as in, where can Utilities store data? In a database/XML file (e.g. the Notes Utility), on the filesystem (e.g. Files Utility) or in a Version Control System (e.g. Revisioned Files Utility)

Oct 23 23:39:30 <beyonddc> Max, I think that's up to the utilty developer.

Oct 23 23:39:36 <beyonddc> We provide them with these options.

Oct 23 23:39:41 <beyonddc> Right?

Oct 23 23:39:51 <J_K9> let's try to focus on requirements and design rather than implementation for now, but yes, SVN is the prime choice atm as the VCS backend

Oct 23 23:39:57 <J_K9> yes, that's right - those were just examples :)

Oct 23 23:40:33 <J_K9> But what do you think of database/XML? are XML files a suitable replacement for a database? it would definitely simplify implementation on the client side IMHO

Oct 23 23:40:47 <J_K9> and reduce overheads of running Mira (CPU cycles, etc)

Oct 23 23:41:02 <beyonddc> The database would be running on the server machine anyway.

Oct 23 23:41:08 <beyonddc> Client will just connect to it.

Oct 23 23:41:20 <J_K9> yes, but the data must also be available on the clients for offline use

Oct 23 23:41:32 <fredm> seems like filesystem is a good metaphor - what utilities can you think of that would rather have a db looking api?

Oct 23 23:42:01 <J_K9> the Notes Utility, Poll Utility, Calendar?

Oct 23 23:42:14 <beyonddc> Let say an address book.

Oct 23 23:42:20 <J_K9> the DB/XML backend is quite important

Oct 23 23:42:40 <J_K9> filesystem can be useful if someone is getting creative with Utilities, and the same goes for VCS :)

Oct 23 23:43:00 <J_K9> but DB/XML backend will probably be the most used

Oct 23 23:43:45 <J_K9> which brings us to the question… Database or XML files to store the data for these Utilities? XML is just as capable, has fewer overheads, but is more intensive to read and write to

Oct 23 23:44:11 <beyonddc> I'm not familiar with the database stuff. If data retrieve from the database can convert into XML and stored in client filesystem then yes, that will be great.

Oct 23 23:45:07 <beyonddc> oh… sorry, I misunderstood you Max. You mean just replace the database with XML all together?

Oct 23 23:45:08 <J_K9> that could be an option, but is the database→XML and XML→database transformation worth it? would that not just be a waste of CPU cycles, by using two different systems for the same backend idea?

Oct 23 23:45:12 <J_K9> yes :)

Oct 23 23:45:17 <J_K9> just an idea

Oct 23 23:45:35 <fredm> have you considered adopting something like sleepycat api? that would present an api and allow you to defer the actual physical layer until later

Oct 23 23:46:30 <J_K9> hmm… i'm not too familiar with sleepycat, but we do want to provide a level of abstraction in that these three backends are supported but the Utility developer is free to choose which one to use

Oct 23 23:46:36 <beyonddc> is that a JAVA API? Never heard of sleepycat and I just googled it and it said JAVA.

Oct 23 23:47:00 <J_K9> redirects to Berkeley DB..

Oct 23 23:48:07 <fredm> yikes. i didn't realize they were bought by oracle

Oct 23 23:48:15 <beyonddc> Yes, lol

Oct 23 23:48:19 <J_K9> hehe

Oct 23 23:48:28 <fredm> i'm not sure its java only - there must be others though (not java)

Oct 23 23:48:38 <J_K9> what is the concept behind it, though?

Oct 23 23:49:15 <clsk> hm what's the overhead you say databases create?

Oct 23 23:49:39 <J_K9> it's another service that needs to be running on the system, whereas XML files are easy to read/write to and provide similar functionality

Oct 23 23:50:20 <beyonddc> I don't think XML will make it though because there's too much differences between what you can do with a relational database and xml.

Oct 23 23:50:37 <clsk> yea but database are generally faster.

Oct 23 23:50:42 <fredm> think of it as a flexible persistence layer that acts like a db, but no sql, etc.

Oct 23 23:51:18 <fredm> berkeley db does have a java version, but I bet there are other bindings as well.

Oct 23 23:51:39 <fredm> there are other os solutions maybe too

Oct 23 23:51:48 <fredm> os → open source

Oct 23 23:52:01 <J_K9> clsk - yes, that's true

Oct 23 23:52:27 <beyonddc> Is database capability something that we need for the alpha version?

Oct 23 23:52:33 <fredm> iow, db is a good api for utilities that you mentioned without having to go sql

Oct 23 23:52:55 <J_K9> thanks fredm, that's a good idea

Oct 23 23:53:24 <J_K9> beyonddc - no, it's not, but it'll be good to have down as a requirement, especially if I'm going to talk to the outsourcer about these modules in more detail tomorrow :)

Oct 23 23:54:02 <beyonddc> What kind of stuff you're planning to have the outsourcer to do first?

Oct 23 23:54:33 <fredm> the api towards the utility is the part you want to nail down - there are some interfaces that allow you to configure the backend as flat, xml or relational. that's what a sleepycat looking solution would provide

Oct 23 23:54:36 <J_K9> I want them to do the core first

Oct 23 23:54:51 <J_K9> which is why I was saying we should discuss requirements in order of implementation

Oct 23 23:55:15 <J_K9> ah, right. we'll have to look at sleepycat in more detail

Oct 23 23:55:48 <beyonddc> Sorry, but I must say that I don't think we can provide a good set of requirements by tomorrow.

Oct 23 23:56:08 <fredm> i'll bet there are several c++ binding projects around; some crappy, some not.

Oct 23 23:56:27 <J_K9> neither do I, beyonddc - but we can provide a good enough one to give them a rough idea of the module's requirements :)

Oct 23 23:56:43 <J_K9> knowing Open Source, there must be! we've just got to find them

Oct 23 23:57:25 <J_K9> ok, so requirements are:

Oct 23 23:57:37 <J_K9> - support FS backend

Oct 23 23:57:45 <J_K9> - support DB[-like] backend

Oct 23 23:57:58 <J_K9> - support version controlled backend

Oct 23 23:58:04 <clsk> I think the main thing that they should work on is solving the problem of how we're going to syncronizate data between the different users

Oct 23 23:58:08 <J_K9> - abstraction layer on top of these

Oct 23 23:58:15 <J_K9> clsk - isn't that part of the network module?

Oct 23 23:58:28 <clsk> network and storage

Oct 23 23:58:31 <J_K9> which module, exactly, deals with diffing the systems?

Oct 23 23:58:36 <J_K9> ah, ok. a combination.

Oct 23 23:58:36 <clsk> both actually

Oct 23 23:59:50 <J_K9> hmm… that's going to take quite a lot of detail when discussing its design and implementation

Oct 24 00:00:17 <J_K9> let's assume that the Mira Client daemon which checks for updates/modifications to Workplace data is left for a much later release

Oct 24 00:00:43 <clsk> that's their job ;)

Oct 24 00:00:55 <J_K9> hehe

Oct 24 00:01:20 <J_K9> we don't want to give them the excuse of spending a lot of time working on the blueprints to waste our resources though :P

Oct 24 00:01:37 <J_K9> that's why I want to write as much of them as I can myself

Oct 24 00:01:43 <J_K9> with your help, of course :)

Oct 24 00:02:16 <beyonddc> We must start providing work for the outsourcer tomorrow?

Oct 24 00:02:24 <J_K9> no

Oct 24 00:02:29 <clsk> I really don't even know where yo begin. I haven't done anything regarding file-transfer before. So I'd have to do some research.

Oct 24 00:03:05 <J_K9> hmm… well, i was thinking md5 sums, but that could take a lot of number crunching (i.e. many CPU cycles) if there is a lot of data

Oct 24 00:04:01 <beyonddc> It might be a good idea to form like a group of 2 and discuss about each module and then write down some requirements/blue prints then present it to everyone in our team. Lastly provide it to outsourcer.

Oct 24 00:04:15 <clsk> hm well that's after you transfer the file.

Oct 24 00:04:16 <clsk> You have to transfer the files first

Oct 24 00:04:39 <J_K9> well, it depends what stage we're talking about

Oct 24 00:05:01 <J_K9> when the client is syncing to the server just after coming online, it needs to check that local data matches remote data

Oct 24 00:05:28 <fredm> sctp? i'm lazy and want to write the least new code possible. ;)

Oct 24 00:05:33 <J_K9> there also needs to be a system in place to upload local modifications and download remote modifications, taking into account the difference..

Oct 24 00:05:35 <J_K9> lol

Oct 24 00:05:46 <J_K9> well, we don't want to reinvent the wheel, do we? ;)

Oct 24 00:06:13 <J_K9> btw, not to plug my idea of XML files again (), but if we were to use that it'd be easier to diff the files and upload the changes

Oct 24 00:06:40 <J_K9> as opposed to modifications to a DB (although, again, I need to take a closer look at sleepycat)

Oct 24 00:06:47 <fredm> until the schema is out of sync between two entities

Oct 24 00:07:39 <clsk> I really don't see XML being such a good idea.

Oct 24 00:07:46 <J_K9> ok, I'll drop it :)

Oct 24 00:07:59 <clsk> XML are regular files and take time to parse

Oct 24 00:08:04 <J_K9> that is true

Oct 24 00:08:20 <clsk> regular text files*

Oct 24 00:08:26 <fredm> xml shouldn't be exposed facing the utilties, i agree

Oct 24 00:09:23 <fredm> but if xml is the implementation behind a really good persistence layer, then it may not be a problem

Oct 24 00:09:40 <fredm> it does diff well, unlike binary

Oct 24 00:10:24 <fredm> ok, i'll drop it too… :)

Oct 24 00:10:30 <clsk> heh

Oct 24 00:10:31 <clsk> I don't think

Oct 24 00:10:32 <fredm> cisk wins ;)

Oct 24 00:10:32 <J_K9> hehe!

Oct 24 00:10:34 <clsk> It might be a good idea

Oct 24 00:10:58 <clsk> s/I don't think/ I don't know

Oct 24 00:11:04 <clsk> so

Oct 24 00:11:06 <clsk> lets keep it open for discussion

Oct 24 00:11:09 <J_K9> ok

Oct 24 00:11:16 <J_K9> oh, the network module will also need to implement STUN, as discuss on Sunday, right?

Oct 24 00:11:30 <clsk> hmm yea

Oct 24 00:11:40 <clsk> I'm going to have to start reading up on that

Oct 24 00:12:02 <clsk> actually first I want to create an interface for the directory module

Oct 24 00:13:18 <J_K9> that's another thing I can ask the outsourcer about too - if they've got any experience with STUN. perhaps the developer could work with you on that, clsk? it's up to you - you get to tell him what to do :P heh

Oct 24 00:13:42 <clsk> hm I don't know.

Oct 24 00:16:19 <J_K9> ok, here's what I've got so far

Oct 24 00:16:23 <J_K9> storage requirements:

Oct 24 00:16:24 <J_K9> [S_REQUIREMENT] support FS backend

Oct 24 00:16:24 <J_K9> [S_REQUIREMENT] support DB/XML backend

Oct 24 00:16:24 <J_K9> [S_REQUIREMENT] support Version Controlled backend

Oct 24 00:16:24 <J_K9> [S_REQUIREMENT] abstraction layer on top of these

Oct 24 00:16:24 <J_K9> [S_REQUIREMENT] [AES?] encryption support on all backends

Oct 24 00:16:24 <J_K9> [S_REQUIREMENT] merging plaintext modifications / flagging up binary/proprietary format modification conflicts

Oct 24 00:16:24 <J_K9> [S_REQUIREMENT] synchronisation between client and server… client updates to server, server merges data, merged server data to client

Oct 24 00:17:04 <J_K9> network requirements:

Oct 24 00:17:05 <J_K9> [N_REQUIREMENT] support communication protocols (as already documented)

Oct 24 00:17:05 <J_K9> [N_REQUIREMENT] file transfer

Oct 24 00:17:05 <J_K9> [N_REQUIREMENT] encryption [SSL?]

Oct 24 00:17:05 <J_K9> [N_REQUIREMENT] STUN

Oct 24 00:17:27 <clsk> STUN for peer-to-peer

Oct 24 00:17:44 <J_K9> from now on, please use the [$id_REQUIREMENT] syntax to make other requirements stand out - makes reading the logs easier. thanks! :)

Oct 24 00:17:54 <J_K9> yes, that's definitely a requirement

Oct 24 00:17:59 <J_K9> i think we should leave it out of alpha though

Oct 24 00:18:17 <J_K9> we can test it in a Class C subnet environment

Oct 24 00:18:18 <clsk> I don't think that should be too hard to implement

Oct 24 00:18:32 <beyonddc> [S_REQUIREMENT] abstraction layer on top of each backend, meaning one set of API per backend.

Oct 24 00:18:34 <J_K9> oh, well, in that case it can be a requirement for 0.1 alpha

Oct 24 00:18:36 <fredm> STUN for server too, if it's outside the firewall

Oct 24 00:18:47 <J_K9> ah, yes, good point

Oct 24 00:18:49 <clsk> specially since there are so many implementations out there

Oct 24 00:19:22 <clsk> hm what do you mean?

Oct 24 00:19:27 <J_K9> beyonddc - have you managed to draw up a list of targets for 0.1 alpha release?

Oct 24 00:19:59 <J_K9> what fredm means is that if the server is NAT'd then STUN on the client end may be required too to connect to the server

Oct 24 00:20:09 <J_K9> although wouldn't ports be forwarded to server/UPnP be working?

Oct 24 00:20:12 <beyonddc> What do you mean by targets? A list of goals we would like to accomplish for alpha release?

Oct 24 00:20:16 <J_K9> oh, is that another requirement - UPnP?

Oct 24 00:20:35 <clsk> hm you need to make a connection to the server first

Oct 24 00:20:38 <J_K9> beyonddc - yes :) it was you who said you were going to work on that, right? or maybe i'm getting confused :)

Oct 24 00:21:00 <clsk> there's no way to find out a NATd computer's address if there isn't any previous regular connection

Oct 24 00:21:03 <clsk> from what I understand

Oct 24 00:21:10 <beyonddc> eh… if I did volunteer it then I must forgot.

Oct 24 00:21:26 <J_K9> hmm, may have been someone else then :)

Oct 24 00:21:31 <clsk> what's UPnP?

Oct 24 00:21:38 <J_K9> universal plug and play

Oct 24 00:21:48 <clsk> hm what does it do?

Oct 24 00:21:49 <beyonddc> um…

Oct 24 00:21:49 <J_K9> disabled on most networks due to security risks it can pose

Oct 24 00:22:17 <J_K9> but it basically means that you don't need to forward ports manually - the server requests that functionality from the router automatically

Oct 24 00:22:31 <clsk> oh

Oct 24 00:22:33 <J_K9> it's quite useful in some situations

Oct 24 00:22:38 <J_K9> but it's not an important requirements

Oct 24 00:22:42 <J_K9> *requirement

Oct 24 00:22:47 <clsk> right

Oct 24 00:22:49 <clsk> maybe for later

Oct 24 00:22:52 <J_K9> aye

Oct 24 00:22:56 <fredm> if the client connects out with tcp, then server would not need STUN

Oct 24 00:23:08 <J_K9> [N_REQUIREMENT] UPnP (later implementation - unimportant)

Oct 24 00:23:43 <J_K9> true, as the session is already set up by the client and confirmed by the server, correct?

Oct 24 00:24:12 <fredm> true for tcp, not for udp - most p2p uses udp though

Oct 24 00:24:26 <fredm> avoids connection overhead

Oct 24 00:24:45 <J_K9> aye, but I don't think we'll be using udp - it would reduce the reliability of the platform

Oct 24 00:25:06 <fredm> how?

Oct 24 00:25:09 <J_K9> it's up to you though, if you think that's the best way to go then by all means implement it like that :)

Oct 24 00:25:26 <J_K9> well, it's stateless, so there's no guarantee that all of the packets have been received by the recipient

Oct 24 00:26:08 <J_K9> so the recipient might receive a corrupt file, which doesn't help us at all.. at least, that's MHO

Oct 24 00:26:17 <fredm> right, but using tcp will reduce the number of clients you can support on a server. i'm thinking out loud here, so not religious about this

Oct 24 00:26:36 <J_K9> that is true

Oct 24 00:26:41 <fredm> each tcp connection takes a file handle

Oct 24 00:27:03 <fredm> scarily small for most default operating systems

Oct 24 00:27:14 <beyonddc> 1024 i believe

Oct 24 00:27:23 <J_K9> concurrent connections?

Oct 24 00:27:35 <beyonddc> no

Oct 24 00:27:42 <clsk> hm

Oct 24 00:27:56 <clsk> should be a little more than that

Oct 24 00:27:59 <beyonddc> 1024 is the default file handle provided for each app on a system

Oct 24 00:28:07 <fredm> total open file handles - big iron servers usually configure more

Oct 24 00:28:12 <beyonddc> oh, maybe our system admin set it to 1024.

Oct 24 00:28:29 <J_K9> hmm.. so how many concurrent connections do you think would be possible using TCP?

Oct 24 00:28:36 <beyonddc> basically any I/O will consume one file handle.

Oct 24 00:28:41 <beyonddc> depends on how many client.

Oct 24 00:28:46 <beyonddc> 1 client per file handle

Oct 24 00:28:48 <clsk> hm that means I'm going to change the network module to use UDP?

Oct 24 00:28:51 <J_K9> if we're targeting both large and small enterprises as main users of our software, we need to take that into consideration

Oct 24 00:28:56 <J_K9> ah

Oct 24 00:29:07 <beyonddc> and writing to a file is also 1 file handle

Oct 24 00:29:12 <J_K9> well, won't that reduce reliability?

Oct 24 00:29:17 <beyonddc> but when you close the file, you'll release the file handle

Oct 24 00:29:21 <J_K9> so that's 512 concurrent connections max.?

Oct 24 00:29:32 <clsk> every file has a file handle

Oct 24 00:29:39 <beyonddc> Right Alan

Oct 24 00:29:43 <clsk> and in unix almost everything is a file

Oct 24 00:29:50 <J_K9> everything is :P

Oct 24 00:29:58 <clsk> windows is different. I'm not very familiar on how windows works internally

Oct 24 00:29:59 <fredm> you probably want to maintain a small state machine for each client, but that's really detailed

Oct 24 00:30:17 <clsk> but that would be through UDP though

Oct 24 00:30:21 <clsk> correct?

Oct 24 00:30:24 <fredm> yes, udp

Oct 24 00:30:30 <clsk> narf

Oct 24 00:30:39 <J_K9> so would TCP or UDP be the best option, in your opinion?

Oct 24 00:30:39 <clsk> So I have to re-do a lot of stuff

Oct 24 00:30:50 <clsk> I'm using TCP right now.

Oct 24 00:30:56 <clsk> I think Fred is right about TCP

Oct 24 00:31:01 <J_K9> too limited?

Oct 24 00:31:05 <clsk> so we should probably go with UDP

Oct 24 00:31:12 <clsk> yes, when it comes to file handles

Oct 24 00:31:18 <J_K9> hmm, ok

Oct 24 00:31:22 <fredm> let's assume tcp, but look at other p2p systems and see if there are any pain points we want to avoid

Oct 24 00:32:05 <fredm> groove uses a udp protocol - but there may be some good advantages to tcp

Oct 24 00:32:26 <J_K9> direct connect uses TCP for data transfers (http://en.wikipedia.org/wiki/Direct_Connect_(file_sharing))

Oct 24 00:32:31 <clsk> tcp is easier to manage, but that's about it I think

Oct 24 00:32:59 <J_K9> but it means fewer concurrent connections, which isn't good for enterprises (it means they have to set up more server to support their employees)

Oct 24 00:33:19 <clsk> nah

Oct 24 00:33:42 <J_K9> oh, is that 1024 file handles per service on the server or just per server?

Oct 24 00:33:43 <clsk> they'd just have to change the max number of file handlers per program

Oct 24 00:33:45 <clsk> or per user

Oct 24 00:33:49 <fredm> stun is for udp

Oct 24 00:33:56 <J_K9> ah

Oct 24 00:33:56 <clsk> I think it's per program

Oct 24 00:34:00 <J_K9> oh, right

Oct 24 00:34:09 <beyonddc> And you can always increase the file handle limit

Oct 24 00:34:18 <clsk> right

Oct 24 00:34:20 <J_K9> so theoretically they could get around the TCP problem by setting up Mira servers on different ports

Oct 24 00:34:30 <J_K9> and we'd still get the reliability of TCP and avoid having to implement STUN

Oct 24 00:34:36 <clsk> now… is there something like STUN for TCP

Oct 24 00:34:40 <fredm> cisk here's the stun rfc - http://www.faqs.org/rfcs/rfc3489.html

Oct 24 00:34:44 <J_K9> it's not required, is it?

Oct 24 00:34:59 <clsk> cool thanks

Oct 24 00:35:00 <fredm> not needed for tcp

Oct 24 00:35:01 <J_K9> because the session is maintained?

Oct 24 00:35:03 <J_K9> aye

Oct 24 00:35:15 <clsk> hm why not?

Oct 24 00:35:50 <J_K9> http://en.wikipedia.org/wiki/Transmission_Control_Protocol#Connection_establishment

Oct 24 00:35:52 <fredm> the two peers setup a true connection that stays up through the life of the session

Oct 24 00:35:59 <J_K9> indeed

Oct 24 00:36:13 <J_K9> and the router will also record this session and so transfer the packets to the correct internal address

Oct 24 00:36:16 <fredm> problem is poking through firewalls

Oct 24 00:36:25 <clsk> but… how could they if the ip is NATd

Oct 24 00:36:32 <J_K9> see ^

Oct 24 00:36:36 <clsk> right

Oct 24 00:36:39 <clsk> that's the problem.

Oct 24 00:36:46 <J_K9> is it?

Oct 24 00:36:51 <fredm> mucho

Oct 24 00:36:52 <clsk> that's the reason why STUN would come in handy

Oct 24 00:36:55 <clsk> or so I thought

Oct 24 00:37:08 <clsk> mucho?

Oct 24 00:37:13 <J_K9> but with TCP it's not required.. isn't that what we've established?

Oct 24 00:37:13 <beyonddc> taht's spanish

Oct 24 00:37:16 <beyonddc> lol

Oct 24 00:37:30 <clsk> heh I know

Oct 24 00:37:30 <fredm> STUN relies on the fact that NAT is willing to receive traffic on a port that has already been outbound

Oct 24 00:37:48 <clsk> oh

Oct 24 00:37:50 <clsk> I see

Oct 24 00:38:41 <J_K9> so shall I add TCP [STUNfree :P] as another N_REQUIREMENT? it allows us to avoid using STUN and its limitations can be avoided

Oct 24 00:38:45 <fredm> so both clients go outbound to the stun service and they're given each others inbound port number to use

Oct 24 00:39:27 <clsk> I see

Oct 24 00:39:29 <fredm> the problem is chat

Oct 24 00:39:41 <J_K9> chat?

Oct 24 00:39:49 <clsk> chat won't be peer-to-peer

Oct 24 00:39:51 <clsk> at least not right now

Oct 24 00:39:52 <J_K9> ah

Oct 24 00:39:57 <fredm> once each peer wants to connect to all of the other peers, you need n2 connections

Oct 24 00:40:08 <clsk> oh

Oct 24 00:40:09 <fredm> ah - ok

Oct 24 00:40:10 <clsk> that

Oct 24 00:40:15 <J_K9> that's a good point…

Oct 24 00:40:22 <fredm> n^2

Oct 24 00:40:40 <clsk> chat will be done through the server.

Oct 24 00:41:02 <J_K9> but remember we said that if the server went down, it would be great to allow the online client to communicate by chat?

Oct 24 00:41:03 <clsk> later we could set-up an irc-like topology

Oct 24 00:41:14 <J_K9> that could be a later requirement though

Oct 24 00:41:26 <clsk> where some peers would function as chat servers

Oct 24 00:41:35 <clsk> and that would only happen when the main server goes down

Oct 24 00:41:58 <J_K9> [N_REQUIREMENT] use TCP for data transfer and client-server communication (no STUN required)

Oct 24 00:42:54 <clsk> hm no

Oct 24 00:43:00 <clsk> we need UDP

Oct 24 00:43:02 <J_K9> do we?

Oct 24 00:43:10 <clsk> so we can bypass firewalls using STUN

Oct 24 00:43:21 <J_K9> but why is that required for client-server?

Oct 24 00:43:30 <J_K9> it is assumed that the client can connect directly to the server

Oct 24 00:43:34 <clsk> not for client-server

Oct 24 00:43:40 <clsk> but for pee-to-peer file transfer

Oct 24 00:43:55 <fredm> if the port you want to use is blocked by firewall, you have to configure one that the sysadm allows

Oct 24 00:44:03 <J_K9> ah, yes, that N_REQUIREMENT was for client-server alone - I should've made that more clear (i.e. not “and” client-server)

Oct 24 00:44:14 <J_K9> fredm - that's possible in mira.conf

Oct 24 00:44:45 <clsk> so should we really use TCP or UDP?

Oct 24 00:44:50 <clsk> what do you guys think?

Oct 24 00:44:52 <fredm> ok - stun is important for implementers that don't want to take support calls :)

Oct 24 00:44:59 <J_K9> lol

Oct 24 00:45:25 <clsk> heh

Oct 24 00:45:27 <J_K9> could P2P be left for a later release, e.g. 0.2 or 0.3?

Oct 24 00:45:41 <clsk> ok

Oct 24 00:45:41 <J_K9> it's a core requirement, but I think it would be easier if we left it out for now, wouldn't it?

Oct 24 00:45:48 <clsk> but still, do we rally want to use TCP or not?

Oct 24 00:45:50 <J_K9> we want to avoid feature creep..

Oct 24 00:45:54 <fredm> so dumb q - clients can transfer files p2p, but the file isn't part of the workspace?

Oct 24 00:46:08 <fredm> this is related to tcp/udp

Oct 24 00:46:11 <clsk> J_K9: actually that's something that I'd like the outsorcers to work on.

Oct 24 00:46:14 <J_K9> the 'files' would be data from the workplace

Oct 24 00:46:21 <J_K9> clsk - sure thing.

Oct 24 00:46:40 <fredm> so my workspace sync may come from a peer, instead of server?

Oct 24 00:46:50 <clsk> riight

Oct 24 00:46:58 <J_K9> but client-client (i.e. P2P) would only happen if one client has the latest version of some files which another client does not have the latest version of (or data, no necessarily files)

Oct 24 00:46:58 <clsk> it'd still be scheduled by the server though.

Oct 24 00:47:03 <clsk> and you'd have to trust that peer

Oct 24 00:47:03 <J_K9> aye

Oct 24 00:47:08 <fredm> then i think you need udp for the clients to connect

Oct 24 00:47:19 <clsk> right

Oct 24 00:47:25 <J_K9> yes, UDP and STUN would be required for client-client

Oct 24 00:47:26 <clsk> that's what I've been trying to say the whole time

Oct 24 00:47:41 <fredm> i retain the right to stay stupid

Oct 24 00:47:47 <clsk> hehe

Oct 24 00:47:55 <beyonddc> Sorry, I must go. Max, can you please send the team an e-mail of the list of requirements we're forming in this meeting?

Oct 24 00:48:03 <J_K9> [N_REQUIREMENT] UDP and STUN required for client-client communication (clsk → outsourcers?)

Oct 24 00:48:11 <J_K9> beyonddc - definitely!

Oct 24 00:48:12 <fredm> k

Oct 24 00:48:17 <beyonddc> Thank you

Oct 24 00:48:20 <beyonddc> I see you guys later

Oct 24 00:48:21 <clsk> hmm

Oct 24 00:48:21 <fredm> see you

Oct 24 00:48:27 <J_K9> thanks for joining! :)

Oct 24 00:48:29 * beyonddc (i=c72ec6ea@gateway/web/cgi-irc/ircatwork.com/x-c5286469864fc481) has left #mira

Oct 24 00:48:34 <clsk> although you tell you the truth I'd like to work on that myself for the experience.

Oct 24 00:48:42 <clsk> But If I do it myself I know it's going to take a while.

Oct 24 00:48:48 <clsk> so yea..

Oct 24 00:49:04 <fredm> you may be able to find implementations to start with, though

Oct 24 00:49:05 <J_K9> maybe you two could work on it together? or if not just give the guy something else to work on ;)

Oct 24 00:49:14 <clsk> well yea that too

Oct 24 00:49:20 <J_K9> true, that'll lower the learning curve

Oct 24 00:49:23 <clsk> I'm sure there's plenty of implementations out there.

Oct 24 00:49:38 <clsk> actually that I can do myself.

Oct 24 00:49:53 <J_K9> great

Oct 24 00:49:59 <clsk> the one thing I'm really worried about and that I know it's going to be hard is…

Oct 24 00:50:16 <clsk> finding out when a client os out of sync

Oct 24 00:50:23 <J_K9> so, once you've implemented the UDP & STUN system, the outsourcer can work on the client-client data transfer system? is that what you mean?

Oct 24 00:50:31 <J_K9> hmm, yes, I agree :(

Oct 24 00:50:31 <clsk> know how much data to send and how to schedule them through the server.

Oct 24 00:50:41 <J_K9> but if groove can do it, so can we

Oct 24 00:50:48 <J_K9> it'll just involve a complex system

Oct 24 00:50:49 <fredm> :)

Oct 24 00:50:53 <J_K9> :D

Oct 24 00:50:58 <clsk> yea

Oct 24 00:51:02 <J_K9> i'll do my best to draw it out, take a pic and upload it

Oct 24 00:51:03 <clsk> I mean it can be done

Oct 24 00:51:11 <clsk> it's just a very complex system.

Oct 24 00:51:11 <J_K9> but it'll be bloody difficult ;)

Oct 24 00:51:16 <clsk> yea

Oct 24 00:51:16 <J_K9> heh

Oct 24 00:51:26 <clsk> I'll try and do some type of design too tomorrow at work.

Oct 24 00:51:37 <J_K9> that'd be great

Oct 24 00:52:19 <J_K9> hmm… I'm trying to think of some more requirements

Oct 24 00:52:31 <clsk> btw max. I fixed that problem last night

Oct 24 00:52:43 <clsk> the seg fault it ended up being something real simple

Oct 24 00:52:46 <J_K9> awesome! I'll run svn up and compile it :)

Oct 24 00:53:09 <fredm> i have a requirement to go home

Oct 24 00:53:19 <J_K9> hehe, of course :)

Oct 24 00:53:21 <clsk> oh

Oct 24 00:53:25 <fredm> i've checked out the source, i'll update and try to build this wek

Oct 24 00:53:32 <clsk> they require you to go home at a certain time?

Oct 24 00:53:32 <fredm> week

Oct 24 00:53:38 <clsk> cool

Oct 24 00:53:38 <J_K9> lol :P

Oct 24 00:53:40 <fredm> not hardly…

Oct 24 00:53:45 <clsk> heh

Oct 24 00:53:49 <clsk> I wish they did that to me

Oct 24 00:53:57 <clsk> :)

Oct 24 00:54:00 <fredm> my mouth is watering, though

Oct 24 00:54:03 <fredm> hungry

Oct 24 00:54:07 <clsk> oh

Oct 24 00:54:09 <J_K9> hehe

Oct 24 00:54:09 <fredm> i'll bert

Oct 24 00:54:12 <fredm> bet

Oct 24 00:54:14 <clsk> I see

Oct 24 00:54:15 <clsk> I'm really tired

Oct 24 00:54:24 <J_K9> 2am over there, right?

Oct 24 00:54:29 <clsk> I'm about to hit the bed

Oct 24 00:54:32 <clsk> yea

Oct 24 00:54:34 <J_K9> it's 1am here :)

Oct 24 00:54:39 <clsk> I have a meeting at 7am

Oct 24 00:54:41 <J_K9> ouch

Oct 24 00:54:52 <fredm> goodnight then - or good morning. see you next time, guys

Oct 24 00:55:03 <J_K9> likewise! thanks for joining! goodnight :)

Oct 24 00:55:05 <clsk> bye Fred

Oct 24 00:55:15 * fredm has quit ()

Oct 24 00:55:59 <J_K9> anyway, you need to get up early and I'm going to get working on the list of requirements and targets for 0.1 :)

Oct 24 00:56:15 <clsk> btw, max

Oct 24 00:56:15 <clsk> that should be the basic requirement for the outsourcers I think.

Oct 24 00:56:22 <J_K9> the latter we can do with launchpad, btw - not sure if you've noticed other projects linking blueprints to release targets

Oct 24 00:56:34 <J_K9> what we've come up with today?

Oct 24 00:56:44 * fredm (n=fred@65.161.183.15) has joined #mira

Oct 24 00:56:49 <J_K9> lol, wb fred :)

Oct 24 00:57:05 <clsk> Just kind of explain how we want the system to work. Use groove as a reference

Oct 24 00:57:05 <clsk> and tell them that the first thing they're going to implement is what I talked about earlier

Oct 24 00:57:05 <clsk> the synchronization part

Oct 24 00:57:11 <fredm> mac os issues ;)

Oct 24 00:57:14 <J_K9> lol

Oct 24 00:57:19 <J_K9> clsk - ok

Oct 24 00:57:25 <fredm> should have used ubuntu, eh?

Oct 24 00:57:36 <J_K9> i'll send the list of requirements to the list and then you can point out which ones they will work on and in what order?

Oct 24 00:57:37 <fredm> see ya'

Oct 24 00:57:39 <J_K9> definitely ;)

Oct 24 00:57:46 <J_K9> ttys!

Oct 24 00:57:56 * fredm has quit (Client Quit)

Oct 24 00:59:12 <clsk> heh

Oct 24 00:59:13 <clsk> well the thing is that it's more of the general idea of synchronization

Oct 24 00:59:17 <clsk> I'll try to think about it more in detail tomorrow

Oct 24 00:59:25 <J_K9> likewise

Oct 24 00:59:29 <J_K9> let's see what we can come up with :)

Oct 24 01:00:04 <J_K9> thanks for joining - I'll email the list soon and you can read it sometime tomorrow :)

Oct 24 01:00:16 <clsk> ok

Oct 24 01:03:10 <clsk> I'll see you tomorrow then