Replication service system and method for directing the replication of information servers based on selected plurality of servers load

Abstract

A system an method for efficiently providing access by a large number of clients to objects located at a large number of information servers. A non-bottleneck solution is provided to sharing load among servers by migrating or replicating objects over from highly loaded servers to less highly loaded servers. Objects that experience low loading are deleted to make room for more highly used objects and to permit make space for new objects. A naming service is provided to provide rapid access to a replica of a requested objects, while avoiding directing access requests to servers from which replicas of requested objects have been deleted. Hierarchical ordering of replication and naming functions permits a variety of particular access methods to be realized.

Claims

What is claimed is: 1. A replication service system for balancing processing load among a plurality of geographically separated information servers serving requests for objects from a plurality of clients, said replication service system comprising a plurality of replicators, each said replicator receiving messages relating to the loading on a respective selected plurality of less than all of said information servers, said replicators each forming differences based on said received messages, said differences relating to the difference in loading between respective ones of said respective selected plurality of servers, and a messaging system for sending messages between replicators in response to said differences, said messages indicating maximum and minimum loading on ones of said selected plurality of said information servers, said messaging system comprising means for sending messages directing the occurrence of a distribution event between selected ones of said information servers, said distribution events comprising at least one of (i) migrating an object from at least a selected one of said information servers to at least another selected one of said information servers, or (ii) copying an object from at least a selected one of said information servers to at least another selected one of said information servers. 2. The system of claim 1, wherein said messaging system comprises means for directing said occurrence of a distribution event when said differences indicate an unbalance loading that exceeds a predetermined threshold. 3. The replication service system of claim 1, wherein said plurality of replicators are arranged in hierarchical tree relation, said tree having a root replicator and a plurality of replicators descending from said root replicator, said plurality of information servers being connected as leaves to terminating branches of said tree. 4. The replication service system according to claim 3, wherein said messaging system further comprises means for sending messages directing the occurrence of a distribution event between selected ones of said information servers. 5. The replication service system according to claim 2, wherein said distribution event is a migration of an object from a more highly loaded server to a less highly loaded server. 6. The replication service system according to claim 2, wherein said distribution event is a replication of an object from a more highly loaded server to a less highly loaded server. 7. The replication service system according to claim 4, wherein at least one of said replicators comprises means for determining the subordinate server having the maximum load, the subordinate server having the minimum load and means for determining whether said maximum load and said minimum load differ by more than a predetermined threshold amount. 8. The replication service system according to claim 7, wherein said messaging system comprises means for sending said message directing a distribution event only when said determining occurs at said root replicator. 9. The replication service system according to claim 7, wherein said messaging system comprises means for sending said message directing a distribution event when said determining occurs at any of said replicators. 10. A method for balancing processing load among a plurality of geographically separated information servers serving requests for objects from a plurality of clients, said method comprising at each of a plurality of replicators, receiving messages relating to the loading on a respective selected plurality of less than all of said information servers, forming differences based on said received messages, said differences relating to the difference in loading between respective ones of said respective selected plurality of servers, and sending messages to at least one other of said replicators in response to said differences, said messages indicating maximum and minimum loading on ones of said selected plurality of said information servers, and sending messages from selected ones of said replicators directing the occurrence of a distribution event between selected other ones of said information servers, said distribution event comprising at least one of (i) migrating an object from at least a selected one of said information servers to at least another selected one of said information servers, or (ii) copying an object from at least a selected one of said information servers to at least another selected one of said information servers. 11. A naming service system for mapping at least one physical name to each symbolic name used by a client in a request to obtain access to an object in one or more information servers in a system of information servers, said naming service system comprising a plurality of redirectors for redirecting a client request to one of a set of information servers storing said requested object, a plurality of distributors for storing information for mapping said symbolic name into the name of a redirector associated with a partition of the symbolic name space, said partitioning of said symbolic names being lexicographical, and means for effecting a distribution event between respective ones of a plurality of said information servers in response to differences in loading between said respective ones of said plurality of servers, and means for updating information available to said redirectors respecting said set of information servers storing said requested object to reflect the occurrence of a distribution event affecting said requested object, said distribution event comprising at least one of (i) migrating an object from at least a selected one of said information servers to at least another selected one of said information servers, or (ii) copying an object from at least a selected one of said information servers to at least another selected one of said information servers. 12. The naming service system of claim 11 wherein said distributors comprise hashing means for determining the redirector to which said request is mapped. 13. The naming service system of claim 11 wherein said distributors comprise Domain Name Service (DNS) servers.
FIELD OF THE INVENTION The present invention relates generally to the field of distributed information systems. More particularly, the present invention relates, in one aspect, to selective replication and distribution of data objects and services between and among a plurality of information systems. Still more particularly, aspects of the present invention relate to distributed systems and methods for efficient provisioning of objects and services to clients in large (including global) networks. BACKGROUND OF THE INVENTION As networked computers and databases, and users of these systems, have proliferated in numbers and geographic spread, interest has grown in efficiently providing access to information objects and services (hereinafter "objects") at host computers or information servers. Presently, for example, thousands of Internet servers provide a very large number of objects to millions of user clients worldwide. These servers are located in many countries around the world and typically at many locations in each country. In other particular cases, network service providers and private corporations locate server nodes at widely separated points in their networks. A particular challenge faced by developers of these networks of servers is that of providing access to a widely distributed set of clients without overloading particular hosts. The overload may occur, e.g., because a server stores objects that are in high demand and/or the server is a repository for large numbers of objects. Meeting this challenge proves especially difficult when the demand for particular objects varies considerably with time. Thus, while a straightforward replication of all objects at all servers would generally improve availability of a particular object to a range of clients, the cost of such replication is prohibitive. In fact, the economics of replication, distribution and storage do not usually permit design for a worst-case predicted demand condition in such large networks. The load on each server is an important consideration in adequately meeting demand from clients for objects; in general, it is desirable to balance the load among servers. In many existing replicated object systems, with relatively few servers, this question is quite tractable: system administrators or a dedicated computer system process can monitor the load of servers and decide on selective replica placement. When the number of servers increases, however, such human or dedicated process cannot be expected to efficiently direct the creation and deletion of replicas. Current object-location techniques used in distributed server networks assume that sets of object replicas are well known to clients. However, when the number and geographic distribution of servers increases to dense national and international proportions this assumption proves unrealistic; the ability of clients to efficiently locate desired objects at servers increases markedly. One possible solution to the scalability problem of the replication service would be to use a localized "greedy" approach. For example, each hosting server might choose another hosting server at random, and perform load comparisons with this other server. If the load difference exceeds a certain distribution threshold d, the less-loaded server would obtain some replicas of objects kept on the higher-loaded server, thus taking up a portion of its load. Due to the randomness of choice, all pairs of servers would be involved. An approach similar to this is presented in "Adaptive load sharing in homogeneous distributed systems," by T. L. Casavant and J. G. Kuhl, IEEE Trans. on Software Eng., vol. 2(14), pp. 141-154, Feb. 1988. The Casavant, et al paper also defines a threshold for which nodes with a load below the threshold are constrained to not initiate load comparison. However, as the number of servers grows, each server must initiate load comparison more frequently. Otherwise, the average time between load distribution events for any given pair of nodes will grow linearly with the number of nodes, as will the lag between load changes and detection of these changes. Since a server has a limit on how frequently it can perform load distribution, this solution is not scalable to large systems. Another approach is to organize hosting servers in a connected graph structure, with neighboring nodes performing pair-wise load distribution. Since the graph is connected the load distribution involves all servers. This technique is similar to an algorithm described in "Simulations of three adaptive, decentralized controlled, job scheduling algorithms," by J. A. Stankovic, Computer Networks, 8, pp. 199-217, August 1984. One difference is that in the Stankovic paper, when a node sends its load data to a neighbor, it includes its information about all other nodes. These techniques also prove to not be scalable as required for large networks. Another important consideration in establishing a global or other large information system is that of a naming service for objects and servers. In general, naming services are used to map a logical name of an object into the physical name of a replica. The main limiting factors for the name service are the number of clients (which determines the number of requests for name resolution), and the number of objects (which determines the size of the name-mapping database). Another factor is the number of requests from hosting servers for updates of name mappings. Name services are well known in the art, including the Domain Name Service (DNS) used in today's Internet and described, e.g., in P. V. Mockapetris, "Domain Names--Concepts and Facilities," Request for Comments 1034, DDN Network Infromation Center, SRI International, November, 1987. Also used in Internet naming is CCITT (now ITU) Recommendation X.500. However, mappings between host name and IP address seldom change in the DNS scheme. Rather, DNS is primarily an append-only database that permits the addition of new host name/IP address mappings; current DNS implementations support little or no dynamic mapping of host name to IP address. Using current DNS naming service techniques to map logical object names to physical replicas, name server responses cached by clients become incorrect much more quickly. Thus, clients must query the name service much more often, greatly increasing the burden on the name service. Weak-consistency schemes for replicating the mapping database such as that described in B. W. Lampson, "Designing a global name service," Proc. of ACM Conf. on Principles of Distributed Systems, pp1-10, 1986 result in many incorrect responses to clients. These incorrect responses result in the use of an incorrect physical name to access an object--with resulting failure and request renewal. World-Wide-Web (Web) syntax and semantics for object names (URLs) are quite different from those in DNS or X.500. Use of DNS or X.500-compliant symbolic names in networks like the Web would require extensive changes in Web browsers. SUMMARY OF THE INVENTION Limitations of the prior art are overcome and a technical advance is made in accordance with the present invention described in illustrative embodiments herein. In one illustrative embodiment, a method is described for achieving the number and placement of object replicas in a network of servers. This result is achieved without the need for bottleneck-causing global decisions. Using the present inventive methods and resulting network, a server network is realized in which creation and deletion of replicas is minimized when a steady demand for all objects persists for an appropriate period of time. Moreover, when such steady demand exists the load can be distributed among servers in a substantially equal manner. In accordance with an illustrative embodiment, the decision on the number and placement of replicas is made within the network. Moreover, the process is dynamic, with replicas being created and deleted as demand and geographic origins of requests change. The illustrative replication is advantageously transparent to end-user clients, except in improved network response. In accordance with an aspect of the illustrative embodiment, no high demand bottlenecks arise which might require an increase of processing power at any node. Rather, as load increases, the number of nodes can be increased to handle the increased demand. This result obtains whether the high demand is based on the number of clients requesting objects increases, or whether the demand arises from an increase in the number of objects. In accordance with one aspect of the present invention, a new naming service is introduced for finding object replicas. Advantageously, the existing DNS name services can be used as a first level of indirection in the new naming service. In other particular contexts, such as when the characteristics of languages for specifying object content or when certain network protocols are used, the illustrative inventive techniques, or their application, may be modified as required to achieve desired results. BRIEF DESCRIPTION OF THE DRAWING The above-summarized description of illustrative embodiments of the present invention will be more fully understood upon a consideration of the following detailed description and the attached drawing, wherein: FIG. 1 is an overall view of an illustrative network system embodiment of the present invention. FIG. 2 shows a replication service arrangement comprising a hierarchy of replicators for use in the network of FIG. 1. FIG. 3 shows an application of the replication service arrangement of FIG. 2. FIG. 4 shows a naming service arrangement for use in the system of FIG. 1. FIG. 5 is a flowchart illustrating a load balancing system and method in accordance with one embodiment of the present invention. FIG. 6 is a flowchart for a modified version of the system and method of FIG. 5. FIG. 7 is a flowchart for a preferred illustrative load balancing arrangement. FIG. 8 is a flowchart showing an illustrative method for deleting a replica. DETAILED DESCRIPTION Illustrative System Overview FIG. 1 shows an illustrative application of the present invention. There, a wide area network 100 comprises a large number of servers 140-i, i-1, 2, . . . M and clients 110-j, j=1,2, . . . N. Servers 140-i host objects publicly accessible to clients 110-j. A node may play a dual role as a server for some objects and a client for other objects. Clients access objects by their names. The name of an object allows a client to infer the identity of hosting server (e.g., its domain name or IP address) and access method (i.e., the protocol to be used to access the server). For illustrative purposes, it will be assumed that object names embed a domain name of the server that hosts the object. One instance of this environment is the Internet, which comprises, inter alia, Web servers and Web browsers. Such servers typically include computers and related equipment for controlling (often very large) databases, while browsers are typically associated with client terminals or (usually smaller) computers, all as is well known in the art. The network of FIG. 1 is a network of information systems, where objects provide access to information, and thus do not change as a result of an access by a client. However, an access may result in extensive processing by the server to compute the result of the client's query. For example, in a geographical database, a request for the map of a specified area in a specified scale may require generation of the map, which typically is an expensive operation in terms of server resources. A load measure exists for each server 140-i in FIG. 1 to allow comparison of load on each server. For example, when the servers operate using the UNIX operating system, the length of the input queue (as measured, e.g., by the output of the uptime command) proves to be convenient for this purpose. In other particular environments, other measures of load may be preferable. In addition, each individual server can estimate the fraction of its total load due to a given object on the server. In typical operation this is accomplished by monitoring resource consumption (e.g., CPU time, 10 operations, etc.) due to requests for individual objects and dividing up the total load between objects in proportion to their consumption. While the illustrative servers 140 in FIG. 1 may be quite different in total system resources, a server with an average queue length of 1.5 is more heavily loaded than a server with an average queue length of 0.8, regardless of the quantity of processing needed at each server to achieve these queue lengths. In achieving purposes of the present invention, the illustrative embodiment of FIG. 1 provides for replicating an object located on a particular server, say server p, on another server, server q. Alternatively an object may be migrated from server p to server q. x p will denote a replication of object x on server p. load(p) denotes the load of node p, and load(p x ) denotes the load on node p due to object x. In general, if x is migrated from node p to node q, the reduction of load on p may not be equal to the increase of load on q, due to difference in processing power of p and q. Typically, there are two ways by which a system can balance the load: directing client requests to less loaded servers (among those with replicas of the requested object), and migrating or replicating objects between servers. Server selection for a particular client request is typically based on the geographic origin of the request (i.e., the closest replica is chosen), but the selection may be based on another criterion, or at random. Thus, the load, is illustratively balanced by replication or migration of objects. The event of migrating an object from server p to server q, or creating a new replica of an object on q by copying it from p, is called a distribution event. Servers p and q are, respectively, the source and the recipient in the distribution event. It is typically desired that when in a network system like that of FIG. 1 the demand for all objects does not change, the system stabilizes into a state in which the loads are distributed equally among all hosting servers. Thus, when the demand for different objects changes and stabilizes at different levels, the system will eventually re-distribute the load so that it is again distributed equally among hosts. However, small changes in the demand for objects should not trigger load re-distribution; otherwise, the system will hardly ever be stable. It proves convenient to consider a mechanism for load balancing to be stabilizing if there exist two constants, demandDiff and loadDiff such that if the variation in time of the request rate for every object x stays within demandDiff, the system eventually reaches a state where no replicas are created or dropped, and the difference between the load of any two hosting servers does not exceed loadDiff. Likewise, it proves advantageous to avoid load balancing of a type in which isolated regions are created with autonomous load distribution in each region. Thus, a load balancing mechanism desirably avoids a condition in which nodes from different regions have significantly different load, even if individual regions are not in the stable state. More formally, for some L and l<L, and with all nodes partitioned into three sets, A(L,l)={p|load(p)≦L}, B(L,l)={p|l≦load(p)<L}, and C(L,l)={p|load(p)<l}. A mechanism for load balancing is called contiguous if there exist some constants d and t such that for any L and l<L-d, if no node moves between sets A(L,l), B(L,l) and C(L,l) for time t, then there will be a distribution event either with a source in A and recipient outside A, or with a source outside C and recipient inside C. The contiguity criterion may be illustrated by a system with four servers i, j, k, and l. A load balancing mechanism can be derived that balances the load between nodes in node pair i and j, and separately between nodes in node pair k and l, with no balancing performed between the node pairs. The demand for objects hosted by the first pair (i, j) will be assumed to be very high, but unstable (so that there are continuous distribution events occurring between i and j). Also, it will be assumed that there is unstable but overall low demand for objects on k and l. Thus the load on i and j greatly exceeds the load on k and 1, while fluctuating within each pair. This example shows the importance of both the contiguity criterion as well as the stabilization criterion. Returning to the system of FIG. 1, when an object is created, it is placed on one of the hosting servers and is registered with a name service 120. Registering of a related type is known in a so-called "persistent URL" proposal for use in the World Wide Web (hereinafter "Web") operation in the Internet, but such registering of persistent URLs is not part of the present invention. This registration in accordance with the illustrative embodiment involves sending a message to name service 120 informing that service of the physical name of the object and assigning it a symbolic name. The physical name of an object is the name by which the object can be accessed by the clients (in the Web context, this would be the object's URL). The symbolic name uniquely identifies the object. However, it resolves into the name service identity rather than the identity of the server hosting the object. The symbolic name is advertised to the users; it is the name known to the clients. When a client wants to access the object, it uses its symbolic name. Since this name resolves into the name service identity, the request for the object actually arrives at the name service 120 as shown by the link between typical client 110-1 and name service 120. Name server 120 finds a corresponding physical name (due to replication, a symbolic name can map into multiple physical names) and sends it as a special "re-direct" response to the client. The client, such as 110-1 in FIG. 1 then uses the physical name received to access the object at the identified server, such as 140-1 in FIG. 1. Asynchronously, hosting servers periodically report their load to the replication service unit 130. The replication service 130 uses these reports to migrate or replicate objects so that the load is shared among all hosting servers, and replicas are placed close to a majority of requests for the corresponding objects. When a hosting server 140-j creates or drops a replica of an object, it records this fact at the name service 120. Thus, the name service keeps a mapping of the symbolic name of an object to a set of physical names of currently available replicas. When resolving a symbolic name, the name service chooses one of the current set of physical names, applying a fair procedure and taking into account the geographical location of the requesting client. The Replication Service One potential bottleneck in the system of FIG. 1 is the replication service. Indeed, as the number of hosting servers and hosted objects increases, the number of load reports the replication service must process grows, and so does the search space for deciding on replica placement. This section describes a scalable distributed architecture for the replication service 130. Scalability is of interest to the present discussion because important target networks are assumed to be large. The following description of a policy for replica placement is based on the objective of load distribution. A challenge in designing such a policy is that the policy must ensure globally desirable system behavior (according to criteria such as the stabilizing and contiguity criteria discussed above) without creating any bottleneck-producing global decision points. Those skilled in the art will develop other detailed policies and heuristics within the scope of the presently described architecture and infrastructure. An illustrative structure for the replication service 130 of FIG. 1 is shown in FIG. 2. There, it will be seen that replication service 130 comprises a hierarchy of replicator servers. Each replicator server in FIG. 2 comprises standard network computer functionality, including a central processing unit, main and auxiliary memory and input/output facilities, all arranged to receive information from information servers and other replicators to perform, inter alia, analyses of load information, and to generate control messages relating to these analyses and to creation and deletion of object replicas. The highest level replicator is at node 210 and replicators 220-j appear at nodes directly below node 210. Only two levels are shown explicitly in the hierarchy of FIG. 2, but other levels of replicators will appear as required for particular networks at respective nodes at other levels in the hierarchy. For very large networks, the level of such nodes may extend over many such levels in a hierarchy. Direct descendants of a node in the hierarchy of replicators in FIG. 2 are referred to as subordinates. For a server S, let the hosting set H(S) be a set of hosting servers in the subtree rooted at S. In particular, for a hosting server p, H(p)={p}. The hierarchical structure of FIG. 2, can be used to implement a variety of load re-distribution methods, as will now be described in more detail. A first solution based on the structure of FIG. 2 is to use the replicator hierarchy only to find the hosting servers with the highest and lowest load. In this solution, hosting servers report their load to the lowest level replicators. These lowest level replicators then choose the servers with the highest and lowest load and pass this information up to their parents. The parents then choose hosting servers with the highest and lowest load, among those reported by their subordinate replicators. Thus, each replicator reports up the information about the hosting servers with highest and lowest load in its hosting set. This process continues until it reaches the root, which will identify the hosting servers with the highest and lowest load in the system and re-distribute the load between them. Since non-root replicators work in parallel, the whole process takes place in logarithmic time. The flowchart of FIG. 5 illustrates this process. Starting at block 500 in the flowchart, high and low load information is read from subordinate nodes as indicated by process block 501. This is continued at block 501 until the decision at block 502 indicates that all immediately subordinate nodes have input their information. A further test is made at decision block 503 to determine if the information has been input to the root node from its subordinate nodes. If the node processing inputs is not the root node, the high and low load information is passed to the next higher node at block 505. This process continues until the root server node is the node receiving the inputs. Then, messages are sent to the high and low load nodes as described above and indicated by process block 504 in FIG. 5 to cause a distribution event to occur. This solution can be shown to satisfy the stability and contiguity criteria described above. Certain inefficiencies can be identified in the process shown in FIG. 5 and described above that may be important in some network applications. First, the work of lower level replicators whose hosting sets do not contain nodes with the globally highest and lowest load is wasted, even if nodes with the load difference over the threshold were found. Second, the load between only two (or, more generally, a constant number of) hosting servers is re-distributed as the result of an entire sweep of load-reporting messages through the whole hierarchy. Thus, many sweeps may be required before the load is balanced. A more efficient variant of the solution shown in FIG. 5 and described above employs a replicator at each level to distribute load between hosting sets of its subordinates, as illustrated in the flowchart of FIG. 6. Sending of such offload messages at each, or selected ones of, replicators at different level of the hierarchy of replicator provides important advantages in some implementations of the present invention. It will be noted below in connection with the process summarized in the flowchart of FIG. 7 that offload messages can be sent at various replicator levels to subordinates when predetermined load imbalances are recognized. A Preferred Load Balancing Protocol Another system and method solution related to the above-described load balancing solutions that has been found to be particularly advantageous in a variety of network contexts is illustrated in FIG. 3. In this alternative and preferred embodiment, each replicator distributes load among subtrees rooted at its subordinates, and each subordinate accomplishes load distribution within its subtree. This solution will now be described in greater detail, initially with reference to FIG. 3. In accordance with this illustrative preferred embodiment, a protocol is employed which uses three parameters. One of these parameters is a system-wide parameter, the distribution threshold, d, which reflects the load difference between servers required to trigger distribution events. Another parameter, the deletion threshold, u, is host-specific. A hosting server p deletes replicas (except for the last replica) of an object such that load(x p )<u p . As discussed above, the same rate of requests for the same object may result in different values of the object load at different servers. However, u p is adjusted for every server such that if object x qualifies for deletion on one server under certain request rate, it would qualify for deletion on all servers under about the same rate. The third parameter employed by this illustrative protocol is a system-wide stability factor s that reflects the level of variation in server load that should not cause distribution events. The three parameters are constrained by a condition 2*u max +s<d, where u max is the maximum deletion threshold across all hosting servers. Each replicator is advantageously arrranged to have at least three subordinates. In accordance with the illustrative protocol, a hosting server p periodically examines all of its objects, attempting to delete object replicas whose load is below u p . It does not, however, delete the sole replica of an object in the system. It then sends a load report to its parent replicator. Every replicator collects load reports from its subordinates and sends the load report to the higher-level replicator. The load report from node S (either a hosting server or a replicator) has a form (avLoad S , H(S), p max ,S, p min ,S, load max ,S, load min ,S), where avLoad S is the average load of hosting servers from H(S). Thus in the case of a hosting server, the average load is the actual load. H(S) includes information about the number of these servers; p max ,S and p min ,S are the identities of hosting servers chosen among H(S). If S is a leaf, both p min ,S and p max ,S are the identity of S itself. When S is a leaf, i.e. a hosting server, load max ,S and load min ,S are the same and equal to the actual load of S. For a non-leaf server S, p max ,S, p min ,S, load max ,S and load min ,S are calculated in the protocol based on reports from subordinates of S. Upon collecting reports from all subordinates, a replicator R executes the DistributeLoad protocol appearing in Listing 1 attached to this specification. Operations of the routine of Listing 1 will now be described. These operations are illustrated and summarized in the flowchart of FIG. 7. Let S 1 , . . . , S n be subordinates of replicator R, and let S max and S min be subordinates that reported the highest load and the lowest load, respectively. If load max ,Smax exceeds load min ,Smin by more than the distribution threshold, R determines that load re-distribution is needed. It therefore sends a message Offload(p min ,Smin) to node p max ,Smax. Upon receiving this message, node p max ,Smax executes a protocol Offload with node p min ,Smin (described below) to shift some load from p max ,Smax to P min ,Smin. Then, unless R is a root replicator, it calculates and sends its own load report to its parent. R computes the average load of descendant hosting servers and the total number of such servers in the normal way. The rest of the components are calculated depending on whether or not an Offload message was sent. If it was not sent then R passes up load max ,Smax, load min ,Smin, p max ,Smax, p min ,Smin as the corresponding part of its report. If Offload message was sent, load re-distribution will be performed between descendants of S max and S min , which will render invalid their reports sent to R. Therefore, R uses avLoad R as both load max ,R and load min ,R in this case. The motivation for this alternative is that since R is performing load re-distribution in its subtree, it should not expose its parent to load of any subtrees beneath R. Instead, R reports the overall average load of all of its hosting servers. If an ancestor of R will find that this average load differs significantly from the load of another subtree, load re-distribution between some hosting servers of the two subtrees will be performed. As in a previous example, redistribution of load can be accomplished by messages sent at differing levels in the hierarchy of replicators. The block 705 in FIG. 7 illustrates the sending of additional offload messages. Different criteria can be set for this additional redistribution as may be required in particular cases. Thus, R must inform its parent of hosting servers to use, should the parent decide to re-distribute the load. For this purpose, R selects p min ,R and p max ,R. R chooses, among subordinates excluding S max and S min , S' max , and S' min that reported the highest load max and the lowest load min , respectively. Note that since at least three subordinates of R exist, there will always be nodes to choose from. R then sets p max ,R =p max ,S'max and p min ,S'min. The reason for selecting different hosting servers is that p max ,Smax and p min ,Smin are already engaged in load re-distribution. This way, no hosting server will be sent multiple Offload messages in response to the same load report. When hosting server p receives the Offload(q) message, it negotates with q to find object x with the lowest load(x p ) and such that q does not have a replica of x. If no such object x exists, p does nothing. Otherwise, if load(x p ) does not exceed d, x is migrated from p to q; if loads(x) is greater, a replica of x is created on q. Further details of object migration and replication are presented below. The motivation for the preceding procedure is that p attempts to reduce its load by migrating rather than replicating objects, provided it does not create a vicious cycle where an object is migrated back and forth the between p and q. If migration of heavily loaded objects is permitted, q might have become more heavily loaded than p as a result of migration causing x to migrate back, and so on. The constraints on the system parameters are chosen so that, after replication, neither replica of x can be dropped. Otherwise, an infinite cycle could be created. If p has the sole replica of x in the system and load(x p )<d, then if x is replicated on q, both will split this load. As a result, load(x q ) could fall below u q , and x q would be deleted. This would casuse load(x p ) to return to its original value, causing creation of another replica of object x on q, and so on. The importance of not basing redistribution decisions only on the average load of servers in the hosting sets of their subordinates is illustrated by the following example. Reference is made to the hierarchical structure of replicators R0, R1, R2, R3 and R4 and servers p1 through p9 in FIG. 3. The values appearing adjacent to each of the R's and p's in FIG. 3 represent the reported average loads for the respective elements. A value for the distribution threshold will be taken as equal to 4. Then, the difference between the average load reported by any two sibling replicators does not exceed the distribution threshold. However, the maximum difference in load between hosting servers exceeds the threshold. In fact, this difference can grow linearly with the height of the tree, thus violating the stability criterion. If the same conditions appearing in FIG. 3 are used to direct the load-balancing embodiment reflected in Listing 1, then a successful redistribution of load will be accomplished. In particular, R3 will report load min ,R3 =load min ,Smin =5. R4 will report load max ,R4 =6, and load max ,R4 =8. Thus R2 will find load max ,Smax -load min ,Smin =6>4, and send Offload(p4) to p9. In addition, R2 will report the average load of all of its host descendants, load max ,R2 =load min ,R2 =5.33 (since it sent the Offload message). At the same time R1 will report load min ,R1 =1 and load max ,R1 =4, because it did not sent the Offload message. Therefore, R0 will discover load max ,Smax -load min ,Smin =4.33>4 and send Offload(p1) to p8. Name Service The name service 120 in FIG. 1 is a gateway to the information systems shown there. It advantageously comprises computer processing and database functionality to supports the following requests: 1. RegisterObject(physicalName). This request typically comes from a system administrator upon creating a new object. In response, the name service returns a symbolic name assigned to this object. Parts of this name can be proposed by the administrator or the author of the object, but the system must ensure that the overall name is unique and that it resolves into the name service identity and access method. 2. GetObject(symbolicName). This request comes from a client. The resulting response after processing by the name service 120 is advantageously a "redirect" message Redirect(physicalName) with the physical name of a replica of the object. 3. CreateReplica(symbolicName, physicalName). This request comes from a hosting server when an additional replica of the object is created. The name service must add the physicalName to the set of physical names that is mapped to symbolicName. The name service must also acknowledge to the sender of the request that the above operation has succeeded. 4. DeleteReplica(symbolicName, physicalName). This request comes from a hosting server when a replica with physicalName is deleted. The name service removes physicalName from the set of physical names corresponding to symbolicName. An exception to this deletion when this is the last physical name in the set, in which case the name service returns a negative acknowledgement to the sender. The name service also sends a positive acknowledgement to the sender of the request if the delete operation has succeeded. With this acknowledgement, the name service includes a resolved-cnt counter, which contains the number of times the name service sent a Redirect message with this physical name. The typical large scale of the system of the presently described illustrative embodiment, together with the short lifetime of data cached at the clients, usually suggests that there must be a very large number of replicas of the name server (potentially on the order of thousands for a global network). However, the high frequency of updates to the mapping database requires that the number of replicas be kept low, since updating a large number of replicas is impractical. To avoid this dilemma, the functionality of the name server is split between two components. One component keeps only non-volatile data and can be highly replicated. The other keeps volatile data, with scalability achieved through partitioning of the database. Then, the number of replicas of this component is dictated only by availability requirements, and can be low (e.g., between one and three replicas of each segment of the database). The structural organization of the name server is shown in FIG. 4, where a plurality of distributors 300-i is shown interconnected with a plurality of redirectors 310-j. The symbolic name space is partitioned among these redirectors based on a selected hash function. While other particular partitionings will occur to those skilled in the art one simple way is to base partitioning on the lexicographical ordering of names, as shown in the in FIG. 4. This hash function is known to distributors 300-i. Note that while the mapping database is updated frequently, the hash function that partitions the namespace among redirectors is very stable. Requests from clients for name resolution arrive at distributors 300-i which distribute these requests to the appropriate redirectors by applying the hash function to the symbolic name in the request. Redirectors 310-j in FIG. 4 generate "redirect" directives with corresponding physical names and send them back to distributors, to be forwarded to the clients. Redirectors can respond directly to clients, but it is usually preferable to preserve a pure client/server (or request/response) approach. If a client is capable of caching the hash function, the client itself can determine which redirector to send its request to, and can then send the request directly to it. A client seeking a desired object in a load-sharing replications systems of the type generally illustrated in FIG. 1 and described in illustrative embodiments herein, must be able to find a distributor to which to send an appropriate request. This can be done using a new mechanism that mimics the DNS approach for locating its root servers (which function in a similar manner to the distributors of FIG. 4.) Such a mechanism requires storing a variable pointing to a distributor at each client. However, it will prove advantageous in some cases to deploy a plurality of systems of the type generally illustrated in FIG. 1. With a plurality of such systems deployed (e.g., in an Internet context), each system will have a respective set of distributors of the type shown in FIG. 4. Since a client should be able to access objects hosted by all such systems, the mimiced-DNS approach provides that, when a new system is deployed, all existing client sites add a pointer to one of the distributors of the new system. However, given the typically large number and diversity of clients, this often makes deploying a new system very difficult. Equally difficult in systems of large size is changing the set of distributors a given system. A preferred alternative to recreating certain aspects of the existing DNS system in realizing the functionality of a system like that described above and shown in FIGS. 1 and 4, is to rely on existing DNS infrastructure to locate distributors. Since an object name determines a DNS name of the server to which the request for the object should be sent, the client will go to that DNS server to resolve it into an IP address. It proves advantageous for this DNS server to map the object name into a set of IP addresses of distributors. In response to a client's query, the DNS server will send the IP address of a distributor to the client. In a preferred embodiment the IP address sent to the client is that of a server that is geographically closest (or otherwise has the lowest connection cost) to the client. H. W. Braun and K. C. Clafy, "An experimental means of providing geographically oriented responses relative to the source of domain name server queries," Technical Report, San Diego Supercomputing Center, April, 1994 describes an implementation of DNS name resolution based on the origin of the requester. This paper is hereby incorporated by reference as if set out herein in its entirety. Note that the described use of DNS services in the context of the present inventive embodiments conforms to the assumption underlying the existing DNS, viz., the mapping of DNS names into IP addresses changes slowly. Unlike sets of replicas for a given object, the set of distributors changes infrequently. Thus, existing caching mechanisms will keep the load on DNS service manageable. Creation and Deletion of Replicas Creation and deletion of replicas must be coordinated with modifications of the name mappings on the name server 120 in FIG. 1. Otherwise, there may be periods of inconsistency between the mapping database on the name server and the actual replica sets. Such inconsistencies may lead to a situation in which the name server resolves a symbolic name to the physical name of a replica that no longer exists. The client would then have to repeat its request from scratch, possibly many times, until the name server resolves the symbolic name into a valid replica. It is possible to avoid such inconsistency by running replica deletion and mapping update as one distributed transaction, and name resolution as another. This, however, can, in some cases, lead to interference of load distribution activity with user requests. The protocols for deleting and creating replicas in accordance with the present embodiment avoid distributed transactions by maintaining an invariant: the set of physical names to which a symbolic name is mapped always forms a subset of currently valid replicas. To this end, when a new replica is created, the mapping on the name server is modified only after the replica has been created. When a replica is to be deleted, the hosting server first requests the name server to exclude this replica from the mapping. Only after obtaining the conformation from the name server that it successfully did so, does the hosting server delete the replica. Object migration is accomplished by replica creation on the recipient node followed by replica deletion on the source node. This ensures that symbolic names are always resolved to valid replicas. There is still a possibility that between the time the client receives the physical name from the name server and the time the client accesses the corresponding replica, the replica may be deleted. This problem is avoided by maintaining two counters, resolved-cnt and satisfied-cnt. resolved-cnt is maintained by the name server for every physical name, and is incremented every time this name is returned to a client. satisfied-cnt is maintained by hosting servers for every replica and is incremented every time the corresponding replica is used to satisfy a client request When a hosting server decides to delete a replica, it first sends a request to the name server to delete the corresponding physical name from the mapping (as described above). With its acknowledgement, the name server returns the value of resolved-cnt for this name. Note that after that, no new client requests will be directed to this replica because its physical name is excluded from the name mapping. If resolved-cnt received by the hosting server is equal to satisfied-cnt of the replica, then all client requests that the name server directed to this replica have been satisfied and the replica can be safely deleted. Otherwise, replica deletion is delayed until both counters become equal, up to a predetermined timeout, after which the system assumes that the client died or killed the request in progress. and removes the replica. Should the request arrive after the timeout, it will fail--and the client will repeat the entire request (starting with the symbolic name resolution) from scratch. An advantage of this approach over the naive uncoordinated replica deletion is that this event is highly unlikely. This process is illustrated and summarized in the flowchart of FIG. 8. Another Example System A further example using the above-described name service organization and infrastructure will now be described for the case of a World-Wide-Web (Web) information system. The system contains Web servers and Web browsers (clients). Web servers host objects and provide access to them by making object names (uniform resource locators, or URLs) known to the users. A URL for an object directly embeds the identity of its hosting server and access method (the protocol to be used by the browser to access the object). For example, a URL http://www-db.marketing.abcd.com/foo.html embeds the domain name of the host that serves the object (www-db.marketing.abcd.com), the protocol to be used to access the object (http), as well as the object of interest (foo.html). Web objects may form a hyper-text system, with one object containing links to other objects. These links are implemented by including URLs of other objects in the object body. Objects can be passive (e.g. text or image pages) or active. When an active object (typically referred to as a CGI script) is accessed, an executable module runs on the server and dynamically generates data to be returned to the client. Clients obtain objects by connecting to the host (www.marketing.abcd.com.) and requesting the object (foo.html). Many Web browser implementations are in use today (e.g., Mosaic and Netscape browsers. etc). In the current example, when an object foo.html is created, it is placed on one of the hosting servers, for instance, server.abcd.com. Thus, its physical URL becomes http://server.abcd.com/foo.html. Then the object is registered with the system name service, which assigns a symbolic URL (e.g., http://name-server.abcd.com/symb-foo) to the object. It is the symbolic URL of the object that is advertised to the users. Since the client uses the symbolic URL, its requests for the object arrive at name server name-server.abcd.com rather than the hosting server. (Recall that multiple name servers may use the same DNS name name-server.abed.com.) This request triggers the execution of the system script that maps symb-foo to a physical URL http://server.abcd.com/foo.html. The physical URL is then sent to the client via the redirect message, which is part of the standard HTTP protocol that existing Web browsers understand. The client then uses that physical URL to obtain the object in the usual manner. As the hosting server becomes overloaded, the replicator service described above will cause the creation of a replica of the object (replica-foo-1) on another available hosting server, say, server2.abcd.com. This replica's physical URL is http://server2.abcd.com/replica-foo-1. Subsequent requests for the object are redirected to one of the two replicas in a round-robin or other prescribed manner. With suitable accommodations for existing Web procedures (which may not be present in other large information networks), existing Web browsers may be used to locate and access objects for the current example system, thus making the system transparent to the user. These accommodations will be described below. The need for accommodations arises from the following conditions. First, an object may use a partial URL to organize a link to another object that resides on the same server. Partial URLs do not contain hosting server information--existing browsers by convention use the hosting server of the parent object to send the request for an object referenced by a partial URL. Suppose an object,foo.html, has a link to another object, bar.html. Moreover, it will be assumed that bar.html is not registered with the system, so that its physical URL is used. If bar.html resides on the same server, foo.html used a partial URL to refer to it. When the load balancing system of FIG. 1 moves foo.html to server2. abcd.com, the link to bar.html will become invalid; this server does not have such object. To deal with this problem, it proves advantageous to expand the object registration procedure. When the object is registered with the system, the name server requests the hosting server that keeps the object to convert all partial URLs into full URLs. This conversion is accomplished following the same rules as those applied by the browser when it encounters a partial URL. With active objects, a few additional accommodations are sometimes useful. In particular, since the text that is returned to the client is generated dynamically by an executable, partial URLs cannot be resolved and headers cannot be inserted at registration time. However, the "composite URL" and Java-based solutions to the hot list problem work for active objects as well as passive objects. Partial URL problems, can be dealt with by imposing a restriction on the style in which SGI scripts are written. In particular, it proves convienent to require that such scripts use only full URLs in the generated text. Other modifications and variations of the disclosed methods and systems will be apparent upon reading of the present disclosure; some of these will now be briefly discussed. Geographical Considerations For example, the geographic location of clients may be included in the course of determining object replication and migration. For this variant, it proves convenient to group the universe of hosting servers into regions. This may be done on a country, world-wide (global) or other basis. Orbiting satellite servers may also be included, either in terrestrial regions or in one or more satellite-only regions. In any event, each hosting server is assigned to a region. It also proves convenient to seek to have all messages exchanged pursuant to the modified protocol remain of constant length. The region for a replicator is taken as the union of regions of its subordinates. Each hosting server compiles statistics on the geographical origins of requests for objects. With its load report, the hosting server includes the statistics on the n objects that are most frequently requested from outside its region, where n is a system parameter determined by experience with particular networks. The statistics for an included object include, e.g., those for up to m regions that contributed the most to the outside requests. According to the geographical extension to the basic protocols described above, a replicator R migrates or replicates reported objects among its subordinates to minimize the number of outside requests for the subordinates. The replicator then reports up to its parent the n objects reported to R which have the greatest number of requests from outside R's region. For the present geographical variant, it proves convenient to include in a determination of replica placement only immediate subordinate of a replicator, and not leaf servers. Thus replication placement decisions are reported by a replicator down through its subordinates instead of notifying leaf servers. Experience will dictate in particular cases how to resolve any possible conflict between improving load balancing and improving distance or cost factors associated with geography-based placement decisions. Replication Strategies The preceding discussion of methods and system for load balancing and improving geography or distance and message handling costs in information systems has, for simplicity of exposition, adopted an approach that starts with a small number of object replications and creates additional replicas as the demand for objects grows. Alternative embodiments, all within the scope of the present invention will seek to have as many replicas of objects as possible, and to drop or delete replicas when the demand for space grows due to hosting for new objects. Other factors occasioning increased demand for space include the presence of objects that prove to be in very high demand. In any event, those skilled in the art will recognize that the presently disclosed methods and systems permit for the orderly reduction of replicas as required. Another variant for migrating high-demand objects is to associate such objects (including, as appropriate, lower-demand objects associated with one or more "hot" objects) for migration as a group. Those skilled in the art will also recognize the advantages of caching objects or name resolution results to enhance network speed and reduce message traffic. The systems and methods described above, permit caching at many levels, including client caching, proxy cashing (at one or more hierarchical levels) and at the overall system level. ______________________________________Listing 1______________________________________DistributeLoad():/*Executed by replicator R*/ Let S.sub.1, . . . , S.sub.n be the set of subordinates of R. Find S.sub.max such that load.sub.max, Smax = max.sub.i=1,n (load.sub.max,Si) and S.sub.min such that load.sub.min,Smin = min.sub.i=1,n (load.sub.min,Si) If load.sub.max, Smax - load.sub.min,Smin > d, then send Offload(p.sub.min,Smin) to p.sub.max,Smax ; endif If R ≠ root then ##STR1## ##STR2## If Offload message was not sent then load.sub.max,R = load.sub.max,Smax ; load.sub.min,R = load.sub.min,Smin ; p.sub.max,R = p.sub.max,Smax and p.sub.min,R = p.sub.min,Smin ; else load.sub.max,R = load.sub.min,R = avLoad.sub.R ; find S'.sub.max and S'.sub.min such that load.sub.max,S'max = max.sub.i=1,n;i≠Smax,Smin (load.sub.max,Si) and load.sub.min,S'min = min.sub.i=1,n;i≠Smin,Smax (load.sub.min,Si) p.sub.max,R = p.sub.max,S'max and p.sub.min,R = p.sub.min,S'min ; endif send report to parent;endif______________________________________

Description

Topics

Download Full PDF Version (Non-Commercial Use)

Patent Citations (9)

    Publication numberPublication dateAssigneeTitle
    US-4974256-ANovember 27, 1990At&T Bell LaboratoriesLoad balancing and overload control in a distributed processing telecommunications system
    US-5539883-AJuly 23, 1996International Business Machines CorporationLoad balancing of network by maintaining in each computer information regarding current load on the computer and load on some other computers in the network
    US-5588147-ADecember 24, 1996Microsoft CorporationReplication facility
    US-5708812-AJanuary 13, 1998Microsoft CorporationMethod and apparatus for Migrating from a source domain network controller to a target domain network controller
    US-5745683-AApril 28, 1998Sun Microsystems, Inc.System and method for allowing disparate naming service providers to dynamically join a naming federation
    US-5774668-AJune 30, 1998Microsoft CorporationSystem for on-line service in which gateway computer uses service map which includes loading condition of servers broadcasted by application servers for load balancing
    US-5787442-AJuly 28, 1998Microsoft CorporationCreating interobject reference links in the directory service of a store and forward replication computer network
    US-5832219-ANovember 03, 1998Object Technology Licensing Corp.Distributed object networking service
    US-5923837-AJuly 13, 1999Matias; YossiMethod of accessing data using approximate data structures

NO-Patent Citations (3)

    Title
    Baumgariner et al., Global load balancing strategy for distributed computer system, IEEE, Jul. 1988.
    Dandamudi et. al., A hierarchical load sharing policy for distributed systems, IEEE, Sep. 1997.
    Microsoft Press Computer Dictionary, 3rd. Ed., Microsoft Press, Sep. 1997.

Cited By (275)

    Publication numberPublication dateAssigneeTitle
    US-2016011944-A1January 14, 2016International Business Machines CorporationStorage and recovery of data objects
    US-8024395-B1September 20, 2011Gary OdomDistributed processing multiple tier task allocation
    US-2004244007-A1December 02, 2004International Business Machines CorporationWorkload balancing
    US-6516350-B1February 04, 2003International Business Machines CorporationSelf-regulated resource management of distributed computer resources
    US-8280867-B2October 02, 2012Teradata Us, Inc.Identifying database request sources
    US-2006271645-A1November 30, 2006Microsoft CorporationMethod for multicasting a message on a computer network
    US-6421686-B1July 16, 2002International Business Machines CorporationMethod of replicating data records
    US-7409688-B2August 05, 2008International Business Machines CorporationSystem and method for workload balancing by resetting an average queue depth upon the start of the server instance
    US-7080146-B2July 18, 2006Storage Technology CorporationMethod, apparatus and computer program product for workload balancing among multiple communication of paths to a plurality of devices
    US-7640253-B2December 29, 2009Theimer Marvin M, Jones Michael BMethod and system for managing data records on a computer network
    US-8478903-B2July 02, 2013Level 3 Communications, LlcShared content delivery infrastructure
    US-8306952-B2November 06, 2012Sap AgTechnique for managing messages
    US-6941555-B2September 06, 2005Bea Systems, Inc.Clustered enterprise Java™ in a secure distributed processing system
    US-7693959-B2April 06, 2010Massachusetts Institute Of TechnologyContent distribution system using an alternative domain name system (DNS) and content servers
    US-2007156845-A1July 05, 2007Akamai Technologies, Inc.Site acceleration with content prefetching enabled through customer-specific configurations
    US-6463454-B1October 08, 2002International Business Machines CorporationSystem and method for integrated load distribution and resource management on internet environment
    US-2009037925-A1February 05, 2009Bea Systems, Inc.Smart stub or enterprise java bean in a distributed processing system
    US-2011196830-A1August 11, 2011Yonatan Zunger, Alexandre Drobychev, Alexander Kessleman, Vickrey Rebekah C, Dachille Frank C, George DatuashviliSystem and Method for Managing Replicas of Objects In A Distributed Storage System
    US-9298736-B2March 29, 2016Google Inc.Pruning of blob replicas
    US-9703843-B2July 11, 2017Nec CorporationObject placement apparatus, object placement method and program
    US-6947925-B2September 20, 2005International Business Machines CorporationSystem and method for performing lookups across namespace domains using universal resource locators
    US-6928457-B2August 09, 2005Bea Systems, Inc.Duplicated naming service in a distributed processing system
    US-9325802-B2April 26, 2016Microsoft Technology Licensing, LlcHierarchical scale unit values for storing instances of data among nodes of a distributed store
    US-8447837-B2May 21, 2013Akamai Technologies, Inc.Site acceleration with content prefetching enabled through customer-specific configurations
    US-6466980-B1October 15, 2002International Business Machines CorporationSystem and method for capacity shaping in an internet environment
    US-2007005689-A1January 04, 2007Leighton F T, Lewin Daniel MContent distribution system using an alternative domain name system (DNS) and content servers
    US-2011196829-A1August 11, 2011Vickrey Rebekah C, Dachille Frank C, Gheorghita Stefan V, Yonatan ZungerMethod and System for Providing Efficient Access to a Tape Storage System
    US-6964539-B2November 15, 2005International Business Machines CorporationMethod for managing power consumption of multiple computer servers
    US-8060613-B2November 15, 2011Level 3 Communications, LlcResource invalidation in a content delivery network
    US-7111019-B1September 19, 2006Hitachi, Ltd.Data warehouse system and query processing method used in the system, and data collecting method and apparatus for the method, and charging method and apparatus in the system
    EP-1402385-A1March 31, 2004International Business Machines CorporationDynamic deployment of services in a computing network
    WO-02100071-A3October 02, 2003Nokia CorpPerfectionnements apportes et associes a un reseau reparti
    US-6766335-B2July 20, 2004Iona Technologies Plc.Method and system for providing object references in a distributed object environment supporting object migration
    US-7269603-B1September 11, 2007Sprint Communications Company L.P.Enterprise naming service system and method
    US-9843557-B2December 12, 2017Level 3 Communications, LlcSystems and methods for dynamically registering endpoints in a network
    US-7860964-B2December 28, 2010Level 3 Communications, LlcPolicy-based content delivery network selection
    US-2003195996-A1October 16, 2003Bea Systems, Inc.Clustered enterprise JavaTM in a secure distributed processing system
    US-6957247-B1October 18, 2005Ates Gorkem IInternet system
    US-2014143274-A1May 22, 2014Nec CorporationObject placement apparatus, object placement method and program
    US-6735631-B1May 11, 2004Sprint Communications Company, L.P.Method and system for networking redirecting
    US-9848250-B2December 19, 2017Free Stream Media Corp.Relevancy improvement through targeting of information based on data gathered from a networked device associated with a security sandbox of a client device
    US-9706265-B2July 11, 2017Free Stream Media Corp.Automatic communications between networked devices such as televisions and mobile devices
    US-2007282818-A1December 06, 2007Virage, Inc.Network video guide and spidering
    US-2011016090-A1January 20, 2011Microsoft CorporationHierarchical Scale Unit Values for Storing Instances of Data Among Nodes of a Distributed Store
    US-9317524-B2April 19, 2016Google Inc.Location assignment daemon (LAD) for a distributed storage system
    US-9088529-B2July 21, 2015Coho Licensing LLCDistributed multiple-tier task allocation
    US-8862617-B2October 14, 2014Google Inc.System and method for replicating objects in a distributed storage system
    US-9021112-B2April 28, 2015Level 3 Communications, LlcContent request routing and load balancing for content distribution networks
    US-7069276-B2June 27, 2006Hitachi, Ltd.Computer system
    US-6963914-B1November 08, 2005Lucent Technologies Inc.Method and apparatus for retrieving a network file using a logical reference
    US-2011167111-A1July 07, 2011Akamai Technologies, Inc.Method for operating an integrated point of presence server network
    US-8874523-B2October 28, 2014Google Inc.Method and system for providing efficient access to a tape storage system
    US-2012089700-A1April 12, 2012Contendo, Inc.Proxy server configured for hierarchical caching and dynamic site acceleration and custom object and associated method
    US-8938418-B2January 20, 2015Google Inc.Method and system for efficiently replicating data in non-relational databases
    US-9167419-B2October 20, 2015Free Stream Media Corp.Discovery and launch system and method
    US-2003105836-A1June 05, 2003Nec CorporationDevice and method for specifying location of object in distributed object system
    US-6904602-B1June 07, 2005Sprint Communications Company L.P.Method and apparatus for implementing persistence in trader services associated with a computer system
    US-8108507-B2January 31, 2012Akamai Technologies, Inc.Content delivery network service provider (CDNSP)-managed content delivery network (CDN) for network service provider (NSP)
    US-9747322-B2August 29, 2017Google Inc.Storage of data in a distributed storage system
    US-2015113162-A1April 23, 2015International Business Machines CorporationTransaction Distribution with an Independent Workload Advisor
    US-7376736-B2May 20, 2008Akamai Technologies, Inc.Method and system for providing on-demand content delivery for an origin server
    US-2010248683-A1September 30, 2010Level 3 Communications, Inc.Systems and Methods for Third Party Emergency Call Termination
    US-7177934-B2February 13, 2007Sun Microsystems, Inc.Method and apparatus for providing application specific strategies to a JAVA platform including start and stop policies
    US-9026668-B2May 05, 2015Free Stream Media Corp.Real-time and retargeted advertising on multiple screens of a user watching television
    US-8041809-B2October 18, 2011Akamai Technologies, Inc.Method and system for providing on-demand content delivery for an origin server
    US-7502858-B2March 10, 2009Akamai Technologies, Inc.Integrated point of presence server network
    US-7752262-B2July 06, 2010International Business Machines CorporationSlow-dynamic load balancing system and computer-readable medium
    US-8572210-B2October 29, 2013Level 3 Communications, LlcShared content delivery infrastructure and method of generating a web page
    US-2003187915-A1October 02, 2003Xian-He Sun, Kasidit ChanchioCommunication and process migration protocols for distributed heterogenous computing
    US-9584595-B2February 28, 2017International Business Machines CorporationTransaction distribution with an independent workload advisor
    US-9866925-B2January 09, 2018Free Stream Media Corp.Relevancy improvement through targeting of information based on data gathered from a networked device associated with a security sandbox of a client device
    US-8762395-B2June 24, 2014Oracle International CorporationEvaluating event-generated data using append-only tables
    US-6779017-B1August 17, 2004International Business Machines CorporationMethod and system for dispatching client sessions within a cluster of servers connected to the world wide web
    US-8548978-B2October 01, 2013Virage, Inc.Network video guide and spidering
    US-8010648-B2August 30, 2011Microsoft CorporationReplica placement in a distributed storage system
    US-9305069-B2April 05, 2016Google Inc.Method and system for uploading data into a distributed storage system
    US-7953888-B2May 31, 2011Level 3 Communications, LlcOn-demand overlay routing for computer-based communication networks
    US-8886602-B2November 11, 2014Google Inc.Location assignment daemon (LAD) for a distributed storage system
    US-9838758-B2December 05, 2017David HarrisonRelevancy improvement through targeting of information based on data gathered from a networked device associated with a security sandbox of a client device
    US-8291046-B2October 16, 2012Level 3 Communications, LlcShared content delivery infrastructure with rendezvous based on load balancing and network conditions
    US-8073961-B2December 06, 2011Massachusetts Institute Of TechnologyGlobal hosting system
    US-2014074774-A1March 13, 2014Digital Works, Inc.Distributed data base system and data structure for distributed data base
    EP-1351141-A3October 10, 2007Microsoft CorporationVerfahren und System zur Verwaltung von Datensätzen auf einem Rechnernetzwerk
    US-7657501-B1February 02, 2010Teradata Us, Inc.Regulating the workload of a database system
    US-2015135010-A1May 14, 2015Scott Thomas MacQuarrie, Patrick John Philips, Tudor Morosan, Gregory Arthur AllenHigh availability system, replicator and method
    US-2007061440-A1March 15, 2007Akamai Technologies, Inc.Method and system for providing on-demand content delivery for an origin server
    US-8683076-B2March 25, 2014Level 3 Communications, LlcMethod of generating a web page
    US-2008162417-A1July 03, 2008Ncr CorporationWorkload priority influenced data temperature
    US-2009172145-A1July 02, 2009Ajit Kumar Gupta, Richard David Day, Eric Sven-Johan SwildensIntegrated point of presence server network
    WO-0244835-A3August 29, 2002Gregory L GingerichA method and system for software and hardware multiplicity
    WO-0244835-A2June 06, 2002Gingerich Gregory LProcede et systeme d'execution d'une multiplicite d'objets logiciels dans une multiplicite de systemes materiels
    US-6393458-B1May 21, 2002Genrad, Inc.Method and apparatus for load balancing in a distributed object architecture
    US-7620730-B2November 17, 2009Theimer Marvin M, Jones Michael B, Gummadi Krishna PMethod for multicasting a message on a computer network
    US-2011196664-A1August 11, 2011Yonatan Zunger, Alexandre Drobychev, Alexander Kesselman, Vickrey Rebekah C, Dachille Frank C, George DatuashviliLocation Assignment Daemon (LAD) Simulation System and Method
    US-2004111441-A1June 10, 2004Yasushi Saito, Christos KaramanolisSymbiotic wide-area file system and method
    US-8380659-B2February 19, 2013Google Inc.Method and system for efficiently replicating data in non-relational databases
    US-2014164328-A1June 12, 2014International Business Machines CorporationProactive data object replication in named data networks
    US-9560425-B2January 31, 2017Free Stream Media Corp.Remotely control devices over a network without authentication or registration
    US-2005066331-A1March 24, 2005Shuji Inoue, Lee Kuo Chu, Ozdemir Hasan TimucinDynamic resource management for distributed retrieval system for security
    US-2005203986-A1September 15, 2005Bea Systems, Inc.Duplicated naming service in a distributed processing system
    US-6834284-B2December 21, 2004International Business Machines CorporationProcess and system for providing name service scoping behavior in java object-oriented environment
    US-9589456-B2March 07, 2017Free Stream Media Corp.Exposure of public internet protocol addresses in an advertising exchange server to improve relevancy of advertisements
    US-7024429-B2April 04, 2006Nextpage,Inc.Data replication based upon a non-destructive data model
    US-8041818-B2October 18, 2011Akamai Technologies, Inc.Integrated point of presence server network and functionally organized web sites
    US-6377975-B1April 23, 2002Interactive Intelligence, Inc.Methods and systems to distribute client software tasks among a number of servers
    US-2010251258-A1September 30, 2010Fujitsu LimitedRecording medium having load balancing program recorded thereon, load balancing apparatus and method thereof
    US-2008222291-A1September 11, 2008Weller Timothy N, Leiserson Charles EContent delivery network service provider (CDNSP)-managed content delivery network (CDN) for network service provider (NSP)
    US-8543901-B1September 24, 2013Level 3 Communications, LlcVerification of content stored in a network
    US-2008235400-A1September 25, 2008Level 3 Communications, LlcContent request routing and load balancing for content distribution networks
    US-8744997-B2June 03, 2014Google Inc.Pruning of blob replicas
    US-7864769-B1January 04, 2011Juniper Networks, Inc.Multicast packet replication
    US-7167893-B1January 23, 2007Bellsouth Intellectual Property Corp.Methods and systems for processing a plurality of errors
    US-2011270907-A1November 03, 2011Brother Kogyo Kabushiki KaishaInformation communication system, node device, information processing method and computer-readable storage medium
    US-2005188091-A1August 25, 2005AlcatelMethod, a service system, and a computer software product of self-organizing distributing services in a computing network
    US-8924466-B2December 30, 2014Level 3 Communications, LlcServer handoff in content delivery network
    US-2011196900-A1August 11, 2011Alexandre Drobychev, Alexander Kesselman, Vickrey Rebekah C, Dachille Frank C, George DatuashviliStorage of Data In A Distributed Storage System
    US-2010274762-A1October 28, 2010Microsoft CorporationDynamic placement of replica data
    US-6430578-B1August 06, 2002Sun Microsystems, Inc.Name service for network management architecture
    US-2006253435-A1November 09, 2006Itaru Nishizawa, Shinji Fujiwara, Kazutomo Ushijima, Shigekazu InoharaData warehouse system and query processing method used in the system, and data collecting method and apparatus for the method, and charging method and apparatus in the system
    US-6457011-B1September 24, 2002Microsoft CorporationMethod of updating a shared database in a computer network
    US-6397252-B1May 28, 2002Electronic Data Systems CorporationMethod and system for load balancing in a distributed object system
    US-8949849-B2February 03, 2015Telefonaktiebolaget Lm Ericsson (Publ)Method and devices for dynamic management of a server application on a server platform
    WO-02100071-A2December 12, 2002Nokia CorporationImprovements in and relating to a distributed network
    US-2002129149-A1September 12, 2002Kenneth SchulzMethod and system for automatically directing a web user to a selected web server
    US-8572208-B2October 29, 2013Level 3 Communications, LlcShared content delivery infrastructure
    US-8965845-B2February 24, 2015International Business Machines CorporationProactive data object replication in named data networks
    US-2005210033-A1September 22, 2005International Business Machines CorporationSystem and method for performing lookups across namespace domains using universal resource locators
    US-9154942-B2October 06, 2015Free Stream Media Corp.Zero configuration communication between a browser and a networked media device
    US-8560292-B2October 15, 2013Google Inc.Location assignment daemon (LAD) simulation system and method
    US-8315975-B2November 20, 2012Hewlett-Packard Development Company, L.P.Symbiotic wide-area file system and method
    US-7480679-B2January 20, 2009Bea Systems, Inc.Duplicated naming service in a distributed processing system
    US-7444640-B2October 28, 2008Nokia CorporationControlling processing networks
    US-6438652-B1August 20, 2002International Business Machines CorporationLoad balancing cooperating cache servers by shifting forwarded request
    US-9838474-B2December 05, 2017Apple Inc.Distributing and synchronizing objects
    US-2011196828-A1August 11, 2011Alexandre Drobychev, Alexander Kesselman, Vickrey Rebekah C, Dachille Frank C, George DatuashviliMethod and System for Dynamically Replicating Data Within A Distributed Storage System
    US-8838595-B2September 16, 2014Google Inc.Operating on objects stored in a distributed database
    US-7225254-B1May 29, 2007Akamai Technologies, Inc.Performance computer network method
    US-2006031846-A1February 09, 2006Bea Systems, Inc.Clustered enterprise JavaTM in a secure distributed processing system
    US-8448080-B2May 21, 2013International Business Machines CorporationTarget server identification in a virtualized data center
    US-7822871-B2October 26, 2010Level 3 Communications, LlcConfigurable adaptive global traffic control and management
    US-2005228856-A1October 13, 2005Swildens Eric S, Day Richard D, Vikas Garg, Zaide LiuDistributed on-demand computing system
    US-2002194182-A1December 19, 2002Hitachi, Ltd.Computer system
    US-2011196827-A1August 11, 2011Yonatan ZungerMethod and system for efficiently replicating data in non-relational databases
    US-2010198916-A1August 05, 2010Leighton F Thomson, Lewin Daniel MGlobal hosting system
    US-2003163548-A1August 28, 2003Patrick SticklerDistributed network
    US-8615485-B2December 24, 2013Google, Inc.Method and system for managing weakly mutable data in a distributed storage system
    US-2002023173-A1February 21, 2002Bea Systems, Inc.Duplicated naming service in a distributed processing system
    US-2006064478-A1March 23, 2006Level 3 Communications, Inc.Geo-locating load balancing
    US-2007130103-A1June 07, 2007Malone Donna B, Majewski Edward KMethods and Systems for Processing a Plurality of Errors
    US-2008008089-A1January 10, 2008Akamai Technologies, Inc.Optimal route selection in a content delivery network
    US-9716736-B2July 25, 2017Free Stream Media Corp.System and method of discovery and launch associated with a networked media device
    US-8769055-B2July 01, 2014Microsoft CorporationDistributed backup and versioning
    US-9686596-B2June 20, 2017Free Stream Media Corp.Advertisement targeting through embedded scripts in supply-side and demand-side platforms
    US-9576473-B2February 21, 2017Free Stream Media Corp.Annotation of metadata through capture infrastructure
    US-7584131-B1September 01, 2009Ameriprise Financial, Inc.Method for migrating financial and indicative plan data between computerized record keeping systems without a blackout period
    EP-1566940-A1August 24, 2005Alcatel Alsthom Compagnie Generale D'electriciteA method, a service system, and a computer software product of self-organizing distributing services in a computing network
    US-7945693-B2May 17, 2011Level 3 Communications, LlcControlling subscriber information rates in a content delivery network
    US-6751646-B1June 15, 2004Sprint Communications Company L.P.Method and apparatus for implementing CORBA compliant name services incorporating load balancing features
    US-8560639-B2October 15, 2013Microsoft CorporationDynamic placement of replica data
    US-8868508-B2October 21, 2014Google Inc.Storage of data in a distributed storage system
    US-7103645-B2September 05, 2006Massachusetts Institute Of TechnologyMethod and system for providing content delivery to a set of participating content providers
    US-8495694-B2July 23, 2013Virage, Inc.Video-enabled community building
    US-2003105735-A1June 05, 2003Iona Technologies, PlcMethod and system for providing object references in a distributed object environment supporting object migration
    US-2002062332-A1May 23, 2002Kosuke Nogami, Tomohiro Igakura, Toru EgashiraData distribution system and data distribution method
    GB-2385683-AAugust 27, 2003Thirdspace Living LtdDistribution system with content replication
    US-2011196882-A1August 11, 2011Alexander KesselmanOperating On Objects Stored In A Distributed Database
    US-9338227-B2May 10, 2016Level 3 Communications, LlcAutomated management of content servers based on change in demand
    US-2010274765-A1October 28, 2010Microsoft CorporationDistributed backup and versioning
    US-9386356-B2July 05, 2016Free Stream Media Corp.Targeting with television audience data across multiple screens
    US-8903885-B2December 02, 2014Brother Kogyo Kabushiki KaishaInformation communication system, node device, information processing method and computer-readable storage medium
    US-8645517-B2February 04, 2014Level 3 Communications, LlcPolicy-based content delivery network selection
    US-2005256898-A1November 17, 2005Etsutaro Akagawa, Hiroshi FurukawaComputer system including file sharing device and a migration method
    US-8768350-B2July 01, 2014Level 3 Communications, LlcSystems and methods for locating endpoints in a communication network
    US-9020105-B2April 28, 2015Level 3 Communications, LlcSystems and methods for third party emergency call termination
    US-2003177406-A1September 18, 2003International Business Machines CorporationMethod for managing power consumption of multiple computer servers
    US-2005027862-A1February 03, 2005Nguyen Tien Le, Duc Pham, Zhang Pu Paul, Peter TsaiSystem and methods of cooperatively load-balancing clustered servers
    US-7565410-B2July 21, 2009Nokia CorporationDistributed network
    US-8473613-B2June 25, 2013Level 3 Communications, LlcTransparent redirection of resource requests
    US-2003055969-A1March 20, 2003International Business Machines CorporationSystem and method for performing power management on a distributed system
    US-2011082785-A1April 07, 2011Weller Timothy N, Leiserson Charles EContent delivery network service provider (CDNSP)-managed content delivery network (CDN) for network service provider (NSP)
    US-2003195870-A1October 16, 2003International Business Machines CorporationSystem and method for performing lookups across namespace domains using universal resource locators
    WO-0165357-A1September 07, 2001Interactive Intelligence, Inc.Techniques de distribution de la charge de traitement pour des centres d'appels et d'autres systemes de traitement
    US-6826624-B1November 30, 2004International Business Machines CorporationMethod and apparatus for network resource access request redirection
    US-2004174888-A1September 09, 2004Bradford Barry Lynn, Briel Mark C.Path balancing apparatus and method
    US-8352424-B2January 08, 2013Google Inc.System and method for managing replicas of objects in a distributed storage system
    US-2011161858-A1June 30, 2011International Business Machines CorporationTarget server identification in a virtualized data center
    US-2008163249-A1July 03, 2008International Business Machines CorporationSystem for Workload Balancing by Resetting an Average Queue Depth upon the Start of the Server Instance
    EP-2422290-A4September 26, 2012Microsoft CorpPlacement dynamique de données de duplication
    US-7334232-B2February 19, 2008Bea Systems, Inc.Clustered enterprise Java™ in a secure distributed processing system
    US-2005154768-A1July 14, 2005Microsoft CorporationMethod and system for managing data records on a computer network
    US-6446218-B1September 03, 2002B-Hub, Inc.Techniques for maintaining fault tolerance for software programs in a clustered computer system
    US-7117244-B2October 03, 2006Interactive Intelligence, Inc.Techniques for load distribution processing for call centers and other processing systems
    US-9020887-B2April 28, 2015Proofpoint, Inc.Managing the status of documents in a distributed storage system
    EP-2422290-A2February 29, 2012Microsoft CorporationDynamic placement of replica data
    US-7693840-B1April 06, 2010Sprint Communications Company L.P.Method and system for distribution of common elements
    US-7165108-B2January 16, 2007Sun Microsystems, Inc.Method and apparatus for providing application specific strategies to a JAVA platform including load balancing policies
    US-2007282819-A1December 06, 2007Virage, Inc.Network video guide and spidering
    US-8599697-B2December 03, 2013Level 3 Communications, LlcOverlay network
    US-8041762-B2October 18, 2011At&T Intellectual Property I, LpMethods and systems for processing a plurality of errors
    US-2010100593-A1April 22, 2010Hon Hai Precision Industry Co., Ltd.Method and system for providing content service
    US-2002143848-A1October 03, 2002Vladimir Matena, Rahul Sharma, Masood Mortazavi, Sanjeev KrishnanMethod and apparatus for providing application specific strategies to a JAVA platform including load balancing policies
    US-9659031-B2May 23, 2017Google Inc.Systems and methods of simulating the state of a distributed storage system
    US-2004205761-A1October 14, 2004Partanen Jukka T.Controlling processing networks
    US-9116929-B2August 25, 2015Teradata Us, Inc.Workload priority influenced data temperature
    US-2006221988-A1October 05, 2006Hans-Dieter Geyer, Peter Surma, Vera RoosTechnique for managing messages
    US-8296396-B2October 23, 2012Level 3 Communications, LlcDelivering resources to clients in a distributed computing environment with rendezvous based on load balancing and network conditions
    US-2004024807-A1February 05, 2004Microsoft CorporationAsynchronous updates of weakly consistent distributed state information
    US-7840667-B2November 23, 2010Akamai Technologies, Inc.Content delivery network service provider (CDNSP)-managed content delivery network (CDN) for network service provider (NSP)
    US-2011196832-A1August 11, 2011Yonatan Zunger, Alexandre Drobychev, Alexander Kesselman, Vickrey Rebekah C, Dachille Frank C, George DatuashviliLocation Assignment Daemon (LAD) For A Distributed Storage System
    US-2010106808-A1April 29, 2010Microsoft CorporationReplica placement in a distributed storage system
    US-2001032245-A1October 18, 2001Nicolas FodorIndustrial capacity clustered mail server system and method
    US-9591381-B2March 07, 2017Free Stream Media Corp.Automated discovery and launch of an application on a network enabled device
    US-9854330-B2December 26, 2017David HarrisonRelevancy improvement through targeting of information based on data gathered from a networked device associated with a security sandbox of a client device
    US-8281035-B2October 02, 2012Level 3 Communications, LlcOptimized network resource location
    US-9703947-B2July 11, 2017Free Stream Media Corp.Relevancy improvement through targeting of information based on data gathered from a networked device associated with a security sandbox of a client device
    US-8935366-B2January 13, 2015Microsoft CorporationHybrid distributed and cloud backup architecture
    EP-1402385-A4May 16, 2007IbmDynamic deployment of services in a computing network
    US-8577997-B2November 05, 2013Massachusetts Institute Of TechnologyGlobal hosting system
    US-6910052-B2June 21, 2005Apple Computer, Inc.Distributing and synchronizing objects
    US-2004036191-A1February 26, 2004Rodda Philip A., Turner Terry A.Method of making a fireproof panel
    US-8335769-B2December 18, 2012Google Inc.Executing replication requests for objects in a distributed storage system
    US-7284055-B1October 16, 2007Sprint Communications Company L.P.Method and system for network redirecting
    US-7373644-B2May 13, 2008Level 3 Communications, LlcAutomated server replication
    US-9258383-B2February 09, 2016Free Stream Media Corp.Monetization of television audience data across muliple screens of a user watching television
    US-2011196838-A1August 11, 2011Yonatan Zunger, Alexandre Drobychev, Alexandre Kesselman, George Datuashvili, Zia SyedMethod and System for Managing Weakly Mutable Data In A Distributed Storage System
    US-2010274983-A1October 28, 2010Microsoft CorporationIntelligent tiers of backup data
    US-8667065-B1March 04, 2014Gary OdomDistributed multiple-tier task allocation
    US-6484204-B1November 19, 2002At&T Corp.System and method for allocating requests for objects and managing replicas of objects on a network
    US-7065549-B2June 20, 2006Illinois Institute Of TechnologyCommunication and process migration protocols for distributed heterogeneous computing
    US-7929429-B2April 19, 2011Akamai Technologies, Inc.Optimal route selection in a content delivery network
    US-8131696-B2March 06, 2012Oracle International CorporationSequence event processing using append-only tables
    US-7454755-B2November 18, 2008Bea Systems, Inc.Smart stub or enterprise Java™ bean in a distributed processing system
    US-8468245-B2June 18, 2013Level 3 Communications, LlcDelivering resources to clients in a distributed computing environment
    US-2009100133-A1April 16, 2009International Business Machines CorporationSlow-Dynamic Load Balancing System and Computer-Readable Medium
    US-2005120354-A1June 02, 2005Hitachi, Ltd.Information processing system and information processing device
    US-8423517-B2April 16, 2013Google Inc.System and method for determining the age of objects in the presence of unreliable clocks
    US-9088599-B2July 21, 2015Level 3 Communications, LlcRegistration redirect server
    US-8554724-B2October 08, 2013Google Inc.Method and system for efficiently replicating data in non-relational databases
    US-2002144002-A1October 03, 2002Vladimir Matena, Rahul Sharma, Masood Mortazavi, Sanjeev KrishnanMethod and apparatus for providing application specific strategies to a JAVA platform including start and stop policies
    US-6728770-B1April 27, 2004Storage Technology CorporationMethod and apparatus for workload balancing along multiple communication paths to a plurality of devices
    US-8145765-B2March 27, 2012Akamai Technologies, Inc.Distributed on-demand computing system
    US-2005160429-A1July 21, 2005Heino Hameleers, Frank HundscheidtMethod and devices for dynamic management of a server application on a server platform
    US-8166096-B1April 24, 2012Gary OdomDistributed multiple-tier task allocation
    US-8341118-B2December 25, 2012Google Inc.Method and system for dynamically replicating data within a distributed storage system
    US-7590739-B2September 15, 2009Akamai Technologies, Inc.Distributed on-demand computing system
    US-7475108-B2January 06, 2009International Business Machines CorporationSlow-dynamic load balancing method
    US-9684728-B2June 20, 2017Hewlett Packard Enterprise Development LpSharing video
    US-2004267930-A1December 30, 2004International Business Machines CorporationSlow-dynamic load balancing method and system
    US-8069447-B2November 29, 2011Oracle International CorporationSmart stub or enterprise java bean in a distributed processing system
    US-8112760-B2February 07, 2012International Business Machines CorporationSystem for workload balancing by resetting an average queue depth upon the start of the server instance
    US-7949779-B2May 24, 2011Level 3 Communications, LlcControlling subscriber information rates in a content delivery network
    US-2011196836-A1August 11, 2011Alexander KesselmanExecuting Replication Requests for Objects In A Distributed Storage System
    US-9026554-B2May 05, 2015International Business Machines CorporationProactive data object replication in named data networks
    US-8239530-B2August 07, 2012Akamai Technologies, Inc.Origin server protection service apparatus
    US-7502859-B2March 10, 2009Panasonic CorporationDynamic resource management for distributed retrieval system for security
    US-9203636-B2December 01, 2015Level 3 Communications, LlcDistributing requests across multiple content delivery networks based on subscriber policy
    US-6807567-B2October 19, 2004Nec CorporationData distribution system and data distribution method
    US-2009106411-A1April 23, 2009Akamai Technologies, Inc.Scalable, high performance and highly available distributed storage system for internet content
    US-2008052327-A1February 28, 2008International Business Machines CorporationSecondary Backup Replication Technique for Clusters
    US-2007271280-A1November 22, 2007Oracle International CorporationSequence event processing using append-only tables
    US-2010005175-A1January 07, 2010Akamai Technologies, Inc.Distributed on-demand computing system
    US-9519772-B2December 13, 2016Free Stream Media Corp.Relevancy improvement through targeting of information based on data gathered from a networked device associated with a security sandbox of a client device
    US-2010274982-A1October 28, 2010Microsoft CorporationHybrid distributed and cloud backup architecture
    US-2011196873-A1August 11, 2011Alexander KesselmanSystem and Method for Replicating Objects In A Distributed Storage System
    US-8769049-B2July 01, 2014Microsoft CorporationIntelligent tiers of backup data
    US-9167036-B2October 20, 2015Level 3 Communications, LlcManaged object replication and delivery
    US-2011196822-A1August 11, 2011Yonatan Zunger, Alexandre Kesselman, Alexandre DrobychevMethod and System For Uploading Data Into A Distributed Storage System
    US-7945655-B2May 17, 2011Akamai Technologies, Inc.Scalable, high performance and highly available distributed storage system for internet content
    US-2008263128-A1October 23, 2008Hitachi, Ltd.Information Processing System and Information Processing Device
    US-2011196901-A1August 11, 2011Alexander Kesselman, Alexandre Drobychev, Ford Daniel JSystem and Method for Determining the Age of Objects in the Presence of Unreliable Clocks
    US-7925713-B1April 12, 2011Akamai Technologies, Inc.Method for operating an integrated point of presence server network
    US-2011196833-A1August 11, 2011Alexandre Drobychev, Alexander Kesselman, Vickery Rebekah C, Dachille Frank C, George DatuashviliStorage of Data In A Distributed Storage System
    US-2011196831-A1August 11, 2011Yonatan Zunger, Alexandre Drobychev, Alexandre Kessleman, Vickrey Rebekah C, Dachille Frank C, George DatuashviliPruning of Blob Replicas
    US-2003088660-A1May 08, 2003Bruce Florman, Brown Donald E.Techniques for load distribution processing for call centers and other processing systems
    US-2007100793-A1May 03, 2007Brown Douglas P, Bhashyam Ramesh, Anita RichardsIdentifying database request sources